I create from a json source a csv that I want to use to populate a memsql database with the help of LOAD DATA INFILE.
I have written a typescript script for the conversation and use the library json2csv.
It leaves the values for nulled entries empty though, creating a string like:
foo, bar, , barz, 11 ,
Yet I expect my output to be:
foo, bar, \N , barz, 11 , \N
for my nulled fields. Otherwise, my database will fill in different default values, such as 0 for a number that should be NULL.
I discovered myself doing:
const someEntitites.map((entity: Entity) => {
entity.foo = entity.foo === null ? '\\N' : entity.foo;
entity.bar = entity.bar === null ? '\\N' : entity.bar;
...
return entity;
}
So basically I am hardcoding my approach to my entity, and I also am prone to bug, as I might have forgotten to check a nullable property. And if I am to export another table, I have to repeat this all over again.
How can I generalize this, so I can use this on different entities where the script "discovers" the nullable fields and sets the marker accordingly?
I created a function that iterates over its own properties and sets its value to \N if the according value is null:
const handleNullCases = (record: any): any => {
for (let key in record) {
if (record.hasOwnProperty(key)) {
const value = record[key];
if (value === null) {
record[key] = "\\N";
}
}
}
return record;
};
That way I can reuse that snipplet for other entities as well:
const processedEntities = entities.map(handleNullCases);
const processedEntities2 = entities2.map(handleNullCases);
...
I find it a bit dirty, as that I just typehint for any and cast the value to a string even though it might have been declared as another type.
I'm going to assume all properties in Entity may be null. If so, this typing is a bit safer:
type Nullable<T> = {[K in keyof T]: T[K] | null};
type CSVSafe<T> = {[K in keyof T]: T[K] | '\\N'};
const handleNullCases = <E>(record: Nullable<E>): CSVSafe<E> => {
let ret = Object.assign(record) as CSVSafe<E>;
Object.keys(ret).forEach((key: keyof E) => {
if (record[key] === null) {
ret[key] = '\\N';
}
});
return ret;
};
type Entity = Nullable<{ a: number, b: string, c: boolean, d: number, e: string }>;
const entity: Entity = { a: 1, b: null, c: false, d: null, e: 'e' };
const safeEntity = handleNullCases(entity);
// type CSVSafe<{ a: number; b: string; c: boolean; d: number; e: string; }>
The handleNullCases function will take any object whose values might be null, and return a new object which is just the same except that null values have been replaced with "\\N". The output type will be a CSVSafe<> version of the Nullable<> input type.
Hope that helps.
Related
I've been trying to improve the types of an older project that uses a lot of 'any' whenever things get complicated.
Replit-link
Consider the following irregular data structure where Data is an interface matching the example (some are nested objects, some are not. I use the same mapping function on all pages, depending on what is present in localData):
const data: Data = {
car: { name: 'x', speed: 45},
cat: { fur: true },
random: ['hi', 'bye']
why: "because"
};
Now I'm mapping this data on different pages like so
const nestedKey = 'car';
const localData = {
name: '',
speed: 0
};
Object.keys(localData).forEach(key => {
if (nestedKey && data[nestedKey]) {
// Here I'm not too sure what type to give key to make TS happy
localData[key] = data[nestedKey as keyof Data][key]
} else {
localData[key] = data[key as keyof Data]
}
});
const nestedKey: keyof typeof data = 'car';
keyof typeof data will return this literal type:
'car' | 'cat' | 'random' | 'why'
Which you could consume it like:
localData[key] = data[nestedKey][key]
playground
I have a bunch of interfaces, at least 2-3 levels nested, where some of the leafs are numbers/strings, etc, but others are (numeric) enums.
I don't want to change this.
Now I want to "serialize" objects that implements my interfaces as JSON. Using JSON.stringify is good for almost all cases, but the enums, that are serialized with their (numerical) value.
I know that it's possible to pass a replacer function to JSON.stringify, but I'm stuck, as I'm not sure how to write a function that detect the structure of my object and replace the enum values with the appropriate names.
example:
enum E { X = 0, Y = 1, Z = 2 }
enum D { ALPHA = 1, BETA = 2, GAMMA = 3 }
interface C { e: E; }
interface B { c?: C; d?: D; }
interface A { b?: B; }
function replacer(this: any, key: string, value: any): any {
return value;
}
function stringify(obj: A): string {
return JSON.stringify(obj, replacer);
}
const expected = '{"b":{"c":{"e":"Y"},"d":"ALPHA"}}';
const recieved = stringify({ b: { c: { e: E.Y }, d: D.ALPHA } });
console.log(expected);
console.log(recieved);
console.log(expected === recieved);
It's not possible to automatically find out which enum was assigned to a field, not even with typescript's emitDecoratorMetadata option. That option can only tell you it's a Number and it will only be emitted on class fields that have other decorators on them.
The best solution you have is to manually add you own metadata. You can do that using reflect-metadata node module.
You'd have to find all enum fields on all of your classes and add metadata saying which enum should be used for serializing that field.
import 'reflect-metadata';
enum E
{
ALPHA = 1,
BETA = 2,
GAMMA = 3,
}
class C
{
// flag what to transform during serialization
#Reflect.metadata('serialization:type', E)
enumField: E;
// the rest will not be affected
number: number;
text: string;
}
This metadata could be added automatically if you can write an additonal step for your compiler, but that is not simple to do.
Then in your replacer you'll be able to check if the field was flagged with this matadata and if it is then you can replace the numeric value with the enum key.
const c = new C();
c.enumField= E.ALPHA;
c.number = 1;
c.text = 'Lorem ipsum';
function replacer(this: any, key: string, value: any): any
{
const enumForSerialization = Reflect.getMetadata('serialization:type', this, key);
return enumForSerialization ? enumForSerialization[value] ?? value : value;
}
function stringify(obj: any)
{
return JSON.stringify(obj, replacer);
}
console.log(stringify(c)); // {"enumField":"ALPHA","number":1,"text":"Lorem ipsum"}
This only works with classes, so you will have to replace your interfaces with classes and replace your plain objects with class instances, otherwise it will not be possible for you to know which interface/class the object represents.
If that is not possible for you then I have a much less reliable solution.
You still need to list all of the enum types for all of the fields of all of your interfaces.
This part could be automated by parsing your typescript source code and extracting the enum types for those enum fields and then saving it in a json file that you can load in runtime.
Then in the replacer you can guess the interface of an object by checking what are all of the fields on the this object and if they match an interface then you can apply enum types that you have listed for that interface.
Did you want something like this? It was the best I could think without using any reflection.
enum E { X = 0, Y = 1, Z = 2 }
enum D { ALPHA = 1, BETA = 2, GAMMA = 3 }
interface C { e: E; }
interface B { c?: C; d?: D; }
interface A { b?: B; }
function replacer(this: any, key: string, value: any): any {
switch(key) {
case 'e':
return E[value];
case 'd':
return D[value];
default:
return value;
}
}
function stringify(obj: A): string {
return JSON.stringify(obj, replacer);
}
const expected = '{"b":{"c":{"e":"Y"},"d":"ALPHA"}}';
const recieved = stringify({ b: { c: { e: E.Y }, d: D.ALPHA } });
console.log(expected);
console.log(recieved);
console.log(expected === recieved);
This solution assumes you know the structure of the object, just as you gave in the example.
When parsing a JSON-formatted string I get a linter error:
let mqttMessage = JSON.parse(message.toString())
// ESLint: Unsafe assignment of an `any` value. (#typescript-eslint/no-unsafe-assignment)
I control the content of message so I would like to tell TS that what comes out of JSON.parse() is actually an Object. How can I do that?
Note: I could silence the warning, but I would like to understand if there is a better way to approach the problem.
The problem is that JSON.parse returns an any type.
That's fair enough right - TypeScript doesn't know if it's going to parse out to a string, a number, or an object.
You have a linting rule saying 'Don't allow assigning variables as any'.
So yeah, you could coerce the result of your JSON.parse
type SomeObjectIKnowAbout = {
};
const result = JSON.parse(message.toString()) as SomeObjectIKnowAbout;
What I tend to like doing in this scenario is create a specific parsing function, that will assert at runtime that the obj really is of the shape you are saying, and will do the type casting to you can treat it while you're writing your code as that object.
type SomeObjectIKnowAbout = {
userId: string;
}
type ToStringable = {
toString: () => string;
}
function parseMessage(message: ToStringable ) : SomeObjectIKnowAbout {
const obj = JSON.parse(message.toString()); //I'm not sure why you are parsing after toStringing tbh.
if (typeof obj === 'object' && obj.userId && typeof obj.userId === 'string') {
return obj as SomeObjectIKnowAbout;
}
else {
throw new Error ("message was not a valid SomeObjectIKnowAbout");
}
}
JSON.parse isn't generic, so we can't supply a generic argument to do it.
You have a couple of options.
The simple thing is that since JSON.parse returns any, you can just define the type of what you're assigning it to:
let mqttMessage: MQTTMessage = JSON.parse(message.toString());
(I've used MQTTMessage as a stand-in for the appropriate type.)
That may not be typesafe enough for everyone, though, since it makes the assumption that the string defines what you expect it to define. And it has the problem that if you do it elsewhere, you repeat the assumption.
Instead, you could define a function:
function parseMQTTMessageJSON(json: string): MQTTMessage {
const x: object = JSON.parse(json);
if (x && /*...appropriate checks for properties here...*/"someProp" in x) {
return x as MQTTMessage;
}
throw new Error(`Incorrect JSON for 'MQTTMessage' type`);
}
Then your code is:
let mqttMessage = parseMQTTMessageJSON(message.toString());
As an alternative to type assertions and runtime wrapper functions, you can utilize declaration merging to augment the global JSON object with a generic overload for the parse method. This will allow you to pass through the expected type and give you improved IntelliSense in case you use a reviver when parsing:
interface JSON {
parse<T = unknown>(text: string, reviver?: (this: any, key: keyof T & string, value: T[keyof T]) => unknown): T
}
type Test = { a: 1, b: "", c: false };
const { a, b, c } = JSON.parse<Test>(
"{\"a\":1,\"b\":\"\",\"c\":false}",
//k is "a"|"b"|"c", v is false | "" | 1
(k,v) => v
);
Or, if you are relying on declaration files to augment global interfaces:
declare global {
interface JSON {
parse<T = unknown>(text: string, reviver?: (this: any, key: keyof T & string,
value: T[keyof T]) => unknown): T
}
}
Playground
Is it possible to specify that a field in GraphQL should be a blackbox, similar to how Flow has an "any" type? I have a field in my schema that should be able to accept any arbitrary value, which could be a String, Boolean, Object, Array, etc.
I've come up with a middle-ground solution. Rather than trying to push this complexity onto GraphQL, I'm opting to just use the String type and JSON.stringifying my data before setting it on the field. So everything gets stringified, and later in my application when I need to consume this field, I JSON.parse the result to get back the desired object/array/boolean/ etc.
#mpen's answer is great, but I opted for a more compact solution:
const { GraphQLScalarType } = require('graphql')
const { Kind } = require('graphql/language')
const ObjectScalarType = new GraphQLScalarType({
name: 'Object',
description: 'Arbitrary object',
parseValue: (value) => {
return typeof value === 'object' ? value
: typeof value === 'string' ? JSON.parse(value)
: null
},
serialize: (value) => {
return typeof value === 'object' ? value
: typeof value === 'string' ? JSON.parse(value)
: null
},
parseLiteral: (ast) => {
switch (ast.kind) {
case Kind.STRING: return JSON.parse(ast.value)
case Kind.OBJECT: throw new Error(`Not sure what to do with OBJECT for ObjectScalarType`)
default: return null
}
}
})
Then my resolvers looks like:
{
Object: ObjectScalarType,
RootQuery: ...
RootMutation: ...
}
And my .gql looks like:
scalar Object
type Foo {
id: ID!
values: Object!
}
Yes. Just create a new GraphQLScalarType that allows anything.
Here's one I wrote that allows objects. You can extend it a bit to allow more root types.
import {GraphQLScalarType} from 'graphql';
import {Kind} from 'graphql/language';
import {log} from '../debug';
import Json5 from 'json5';
export default new GraphQLScalarType({
name: "Object",
description: "Represents an arbitrary object.",
parseValue: toObject,
serialize: toObject,
parseLiteral(ast) {
switch(ast.kind) {
case Kind.STRING:
return ast.value.charAt(0) === '{' ? Json5.parse(ast.value) : null;
case Kind.OBJECT:
return parseObject(ast);
}
return null;
}
});
function toObject(value) {
if(typeof value === 'object') {
return value;
}
if(typeof value === 'string' && value.charAt(0) === '{') {
return Json5.parse(value);
}
return null;
}
function parseObject(ast) {
const value = Object.create(null);
ast.fields.forEach((field) => {
value[field.name.value] = parseAst(field.value);
});
return value;
}
function parseAst(ast) {
switch (ast.kind) {
case Kind.STRING:
case Kind.BOOLEAN:
return ast.value;
case Kind.INT:
case Kind.FLOAT:
return parseFloat(ast.value);
case Kind.OBJECT:
return parseObject(ast);
case Kind.LIST:
return ast.values.map(parseAst);
default:
return null;
}
}
For most use cases, you can use a JSON scalar type to achieve this sort of functionality. There's a number of existing libraries you can just import rather than writing your own scalar -- for example, graphql-type-json.
If you need a more fine-tuned approach, than you'll want to write your own scalar type. Here's a simple example that you can start with:
const { GraphQLScalarType, Kind } = require('graphql')
const Anything = new GraphQLScalarType({
name: 'Anything',
description: 'Any value.',
parseValue: (value) => value,
parseLiteral,
serialize: (value) => value,
})
function parseLiteral (ast) {
switch (ast.kind) {
case Kind.BOOLEAN:
case Kind.STRING:
return ast.value
case Kind.INT:
case Kind.FLOAT:
return Number(ast.value)
case Kind.LIST:
return ast.values.map(parseLiteral)
case Kind.OBJECT:
return ast.fields.reduce((accumulator, field) => {
accumulator[field.name.value] = parseLiteral(field.value)
return accumulator
}, {})
case Kind.NULL:
return null
default:
throw new Error(`Unexpected kind in parseLiteral: ${ast.kind}`)
}
}
Note that scalars are used both as outputs (when returned in your response) and as inputs (when used as values for field arguments). The serialize method tells GraphQL how to serialize a value returned in a resolver into the data that's returned in the response. The parseLiteral method tells GraphQL what to do with a literal value that's passed to an argument (like "foo", or 4.2 or [12, 20]). The parseValue method tells GraphQL what to do with the value of a variable that's passed to an argument.
For parseValue and serialize we can just return the value we're given. Because parseLiteral is given an AST node object representing the literal value, we have to do a little bit of work to convert it into the appropriate format.
You can take the above scalar and customize it to your needs by adding validation logic as needed. In any of the three methods, you can throw an error to indicate an invalid value. For example, if we want to allow most values but don't want to serialize functions, we can do something like:
if (typeof value == 'function') {
throw new TypeError('Cannot serialize a function!')
}
return value
Using the above scalar in your schema is simple. If you're using vanilla GraphQL.js, then use it just like you would any of the other scalar types (GraphQLString, GraphQLInt, etc.) If you're using Apollo, you'll need to include the scalar in your resolver map as well as in your SDL:
const resolvers = {
...
// The property name here must match the name you specified in the constructor
Anything,
}
const typeDefs = `
# NOTE: The name here must match the name you specified in the constructor
scalar Anything
# the rest of your schema
`
Just send a stringified value via GraphQL and parse it on the other side, e.g. use this wrapper class.
export class Dynamic {
#Field(type => String)
private value: string;
getValue(): any {
return JSON.parse(this.value);
}
setValue(value: any) {
this.value = JSON.stringify(value);
}
}
For similar problem I've created schema like this:
"""`MetadataEntry` model"""
type MetadataEntry {
"""Key of the entry"""
key: String!
"""Value of the entry"""
value: String!
}
"""Object with metadata"""
type MyObjectWithMetadata {
"""
... rest of my object fields
"""
"""
Key-value entries that you can attach to an object. This can be useful for
storing additional information about the object in a structured format
"""
metadata: [MetadataEntry!]!
"""Returns value of `MetadataEntry` for given key if it exists"""
metadataValue(
"""`MetadataEntry` key"""
key: String!
): String
}
And my queries can look like this:
query {
listMyObjects {
# fetch meta values by key
meta1Value: metadataValue(key: "meta1")
meta2Value: metadataValue(key: "meta2")
# ... or list them all
metadata {
key
value
}
}
}
I have this interface:
interface IParameters {
form: number;
field: string;
...
}
I want the formproperty to be number or function and field to be string or function.
I try something like this:
interface IParameters {
form: number | Function;
field: string | Function;
...
}
I need this because in my code i use this variables like this:
var form = (typeof _oParameters.form === "function" ? _oParameters.form() : _oParameters.form);
var field = (typeof _oParameters.field === "function" ? _oParameters.field() : _oParameters.field);
I don't want change this variable in all my code from string/number to default function and to prevent setting this variables to other types.
but if I try to call one of this two variable like function:
var param:IParameters;
param.form();
...
I get this error:
Cannot invoke an expression whose type lacks a call signature.
but param.form = 12; works.
The only solution that i have found is to declare form and field to any type.
Is other way to define this variable without any type?
If you try to use the code from seriesOne you will notice, that you cannot assign form (or field) a value that isn't a function.
You would get a
Type '{ form: string; }' is not assignable to type 'IParameters'.
Types of property 'form' are incompatible.
Type 'string' is not assignable to type '() => string'.
I found that using form: (() => string) | string; resolves this problem.
You can try the code at the typescript playground.
Working sample:
interface IParameters {
form: (() => string) | string;
}
function strFunct(): string {
return 'Hello TypeScript';
}
function test() {
let paramA: IParameters = {
form: strFunct
}
let paramB: IParameters = {
form: 'Hello stackoverflow'
}
}
class Foo {
constructor(param: IParameters) {
var x = typeof param.form === 'string' ? param.form : param.form();
}
}
Perhaps if you define your union type using a call signature rather than Function, like so...
interface IParameters {
// form is of type function that returns number, or number literal.
form: () => number | number;
// field is of type function that returns a string, or string literal.
field: () => string | string;
}
class Foo {
constructor (param: IParameters) {
var x = typeof param.form === "number" ? param.form : param.form();
var y = typeof param.field === "string" ? param.field : param.field();
}
}
Here, I am still checking form and field using typeof, however TypeScript is happy for it to be either a function call, or a value.
Upd
That behavior appears to be an issue #3812 with TypeScript.
It is to be fixed as a part of TypeScript 2.0 according to the milestone assigned.
Original answer
You could use instanceof as a type guard like
var field = (_oParameters.field instanceof Function ? _oParameters.field() : _oParameters.field);