is there a shorthand way figuring out parameter type based on the json evaluated by readJSON groovy package. I am using resulting event_processor_parameters in a job such as
build job: "dvmt-event-processor-dev", wait: false, parameters: event_processor_parameters
I have this working but I would like to have more cleaner way.
props = readJSON text: env.hb_job_params
for ( param in props.get(application_server)) {
if (param.value.getClass() == Boolean){
event_processor_parameters.add([$class: 'BooleanParameterValue', name: param.key, value: param.value])
}
else if (param.value.getClass() == String){
event_processor_parameters.add([$class: 'StringParameterValue', name: param.key, value: param.value])
}
}
env.hb_job_params ==>
{
"server1": {
"ENV": "DEV",
"dev_xbar_host": "xbarserver1",
"platform_type" : "o2",
"dev_app_host" : "server1",
"VERSION" : "1.0.0.23",
force_build: false
}
}
as a variant:
for ( param in props.get(application_server)) {
def clazz = "${param.value.getClass().getSimpleName()}ParameterValue"
event_processor_parameters.add([$class: clazz, name: param.key, value: param.value])
}
Related
What I would like to achieve is to be able to generate eventgridsubscriptions very easy using bicep. Because manually it costs a lot of time. I have to create like a over a dozen each day.
I have the following bicep file called main.bicep
param eventSubscriptionName string = 'eventSubName'
param storageAccountName string ='storeAccountName'
param deadLetterAccountName string = 'deadlttrstore'
param serviceBusQueueName string = 'queue.name.enter'
param onrampName string = 'storagecontainername'
resource storageAccount 'Microsoft.Storage/storageAccounts#2021-09-01' existing = {
name: storageAccountName
}
resource deadLetterAccount 'Microsoft.Storage/storageAccounts#2021-09-01' existing = {
name: deadLetterAccountName
}
resource serviceBusQueue 'Microsoft.ServiceBus/namespaces/queues#2021-11-01' existing = {
name: serviceBusQueueName
}
resource eventgridsubscription 'Microsoft.EventGrid/eventSubscriptions#2021-12-01' = {
name: eventSubscriptionName
scope: storageAccount
properties: {
deadLetterDestination: {
endpointType: 'StorageBlob'
properties: {
blobContainerName: 'storage-deadletters'
resourceId: deadLetterAccount.id
}
}
destination: {
endpointType: 'ServiceBusQueue'
properties: {
deliveryAttributeMappings: [
{
name: serviceBusQueueName
type: 'Static'
properties: {
isSecret: false
value: ''
}
}
]
resourceId: serviceBusQueue.id
}
}
eventDeliverySchema: 'EventGridSchema'
filter: {
enableAdvancedFilteringOnArrays: false
includedEventTypes: [
'Microsoft.Storage.BlobCreated'
]
isSubjectCaseSensitive: false
subjectBeginsWith: '/blobServices/default/containers/${onrampName}'
subjectEndsWith: '.json'
}
retryPolicy: {
eventTimeToLiveInMinutes: 1440
maxDeliveryAttempts: 5
}
}
}
When I want create the event subscription using az cli with:
az deployment group create -f main.bicep -g <resource-group>
I get the following error:
{
"status": "Failed",
"error":
{
"code": "DeploymentFailed",
"message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
"details":
[
{
"code": "BadRequest",
"message": "{\r\n \"error\":
{\r\n \"code\": \"InvalidTemplate\",\r\n \"message\": \"Unable to process template language expressions for resource '/subscriptions/x1234456-f9cc-44e5-bc40-5f02d962f2d7/resourceGroups/<resource-group>/providers/Microsoft.Storage/storageAccounts/<storage-account>/providers/Microsoft.EventGrid/eventSubscriptions/eventSubName' at line '34' and column '5'. 'The language expression property array index '1' is out of bounds.'\",\r\n \"additionalInfo\":
[\r\n {\r\n \"type\": \"TemplateViolation\",\r\n \"info\": {\r\n \"lineNumber\": 34,\r\n \"linePosition\": 5,\r\n \"path\": \"\"\r\n }\r\n }\r\n ]\r\n }\r\n}"
}
]
}
}
I am working according to the template documented at MS here:
https://learn.microsoft.com/en-us/azure/templates/microsoft.eventgrid/eventsubscriptions?tabs=bicep
Eventually the solution was quite simple the servicebus resource was missing its parent resource namely the servicebus namespace. Once that was added it worked.
resource serviceBus 'Microsoft.ServiceBus/namespaces#2021-11-01' existing = {
name: serviceBusName
}
and
resource serviceBusQueue 'Microsoft.ServiceBus/namespaces/queues#2021-11-01' existing = {
parent: serviceBus
name: serviceBusQueueName
}
to
param eventSubscriptionName string = 'eventSubName'
param storageAccountName string ='storeAccountName'
param deadLetterAccountName string = 'deadlttrstore'
param serviceBusQueueName string = 'queue.name.enter'
param onrampName string = 'storagecontainername'
resource storageAccount 'Microsoft.Storage/storageAccounts#2021-09-01' existing = {
name: storageAccountName
}
resource deadLetterAccount 'Microsoft.Storage/storageAccounts#2021-09-01' existing = {
name: deadLetterAccountName
}
resource serviceBus 'Microsoft.ServiceBus/namespaces#2021-11-01' existing = {
name: serviceBusName
}
resource serviceBusQueue 'Microsoft.ServiceBus/namespaces/queues#2021-11-01' existing = {
parent: serviceBus
name: serviceBusQueueName
}
resource eventgridsubscription 'Microsoft.EventGrid/eventSubscriptions#2021-12-01' = {
name: eventSubscriptionName
scope: storageAccount
properties: {
deadLetterDestination: {
endpointType: 'StorageBlob'
properties: {
blobContainerName: 'storage-deadletters'
resourceId: deadLetterAccount.id
}
}
destination: {
endpointType: 'ServiceBusQueue'
properties: {
deliveryAttributeMappings: [
{
name: serviceBusQueueName
type: 'Static'
properties: {
isSecret: false
value: 'some-value'
}
}
]
resourceId: serviceBusQueue.id
}
}
eventDeliverySchema: 'EventGridSchema'
filter: {
enableAdvancedFilteringOnArrays: false
includedEventTypes: [
'Microsoft.Storage.BlobCreated'
]
isSubjectCaseSensitive: false
subjectBeginsWith: '/blobServices/default/containers/${onrampName}'
subjectEndsWith: '.json'
}
retryPolicy: {
eventTimeToLiveInMinutes: 1440
maxDeliveryAttempts: 5
}
}
}
I tried to reproduce the error message you posted but could get the same result. I did get an error message because value: '':
{
name: serviceBusQueueName
type: 'Static'
properties: {
isSecret: false
value: ''
}
}
When I updated to the following, it worked:
{
name: serviceBusQueueName
type: 'Static'
properties: {
isSecret: false
value: 'some-value'
}
}
The error message I saw with the empty string was:
Null or empty value for static delivery attribute queue-name-enter. Static delivery attribute value must be a non-empty string.
After adding some random text, the deployment completed successfully.
I have some types:
export type Ping = {
kind: 'ping',
lag?: number
}
export type Message = {
kind: 'message',
value: string
}
I have an incoming message string encoded json:
let msg = "{kind:'message', value: 3 }",
ping = "{kind:'ping'}";`;
After I convert this into an object:
let obj = JSON.parse(msg);
I want to validate these messages to have the expected properties and dispatch them like so:
export function isMessage(_: any): _ is Message {
if (typeof _ === 'object') {
let res = (_ as Message);
if (res.kind === 'message' && res.value && typeof res.value === 'string') {
return true;
}
}
return false;
}
export function use(_: any) {
if (isMessage(_)) {
console.log('Message: ', _.value);
}
}
Do I have to typecheck every field of every kind of message like above, or is there an simpler way of doing this?
The simpler, or at least cleaner, way is to write a JSON Schema and run your data through a validator like ajv. Here's how your Message type could be validated:
import Ajv, { JSONSchemaType } from 'ajv';
const ajv = new Ajv();
export interface Message {
kind: 'message',
value: string
}
const messageSchema: JSONSchemaType<Message> = {
type: 'object',
properties: {
kind: { type: 'string', const: 'message' },
value: { type: 'string' }
},
required: ['kind', 'value'],
additionalProperties: false
};
const isMessage = ajv.compile(messageSchema);
export { isMessage };
Hi I'm facing problem with serialization of map where key is an custom class.
data class KeyClass(val id: Int, val name: String) {
fun toJSON() = "\"KeyClass\": {\"id\":$id,\"name\":\"$name\"}"
}
Invocation:
fun method(): Map<KeyClass, List<Something>> = ...
My jackson Serializer ofc I'm also adding this as module in objectMapper:
class KeyClassSerializer : JsonSerializer<KeyClass>() {
override fun serialize(value: KeyClass, gen: JsonGenerator, serializers: SerializerProvider) {
gen.writeRawValue(value.toJSON())
}
}
class KeyClassSerializerModule : SimpleModule() {
init {
addKeySerializer(KeyClass::class.java, KeyClassSerializer())
}
}
And JSON I'm receiving is:
"\"KeyClass\": {\"id\":1,\"name\":\"Thomas\"}" : [Something:...]
I mean the value of map is serialized correctly but key isn't.
I assume the expected result is :
"KeyClass": {
"id": 1,
"name":"Thomas"
} : [...]
But it's not valid Json. You can still do something like :
{
"key" : {
"id": 1,
"name":"Thomas"
},
"value" : [...]
}
Given a JSON output of an api:
{
"id": 13,
"name": "horst",
"cars": [{
"brand": "VW",
"maxSpeed": 120,
"isWastingGazoline": true
}]
}
I would like to define interfaces for typescript:
export interface Car {
brand: string;
maxSpeed: number;
isWastingGazoline: boolean;
}
export interface RaceCarDriver {
id: number;
name: string;
cars: Car[];
}
Yet I don't want them to type them manually, I rather have a script generate them for me.
How can I convert json into typescript interfaces? I also don't want to use webservices like MakeTypes or json2ts.
You can write the script using typescript compiler API and its ability to infer types. I was really surprised how easy it was.
You have to wrap your sample data to make it compile-able as typescript code. The script will pick all variable declarations and try to print inferred types for them. It uses variable names and property names for assigning names to types, and if two objects have a property with the same name, it will pick the type from the first one. So it will not work in case these types are actually different (the fix is left as an exercise). For your JSON output, the data sample will look like
file sample.ts
let raceCarDriver = {
"id": 13,
"name": "horst",
"cars": [{
"brand": "VW",
"maxSpeed": 120,
"isWastingGazoline": true,
}]
};
The script was tested with Typescript 2.1 (just released):
npm i typescript
npm i #types/node
./node_modules/.bin/tsc --lib es6 print-inferred-types.ts
node print-inferred-types.js sample.ts
output:
export interface RaceCarDriver {
id: number;
name: string;
cars: Car[];
}
export interface Car {
brand: string;
maxSpeed: number;
isWastingGazoline: boolean;
}
Here is the script: print-inferred-types.ts:
import * as ts from "typescript";
let fileName = process.argv[2];
function printInferredTypes(fileNames: string[], options: ts.CompilerOptions): void {
let program = ts.createProgram(fileNames, options);
let checker = program.getTypeChecker();
let knownTypes: {[name: string]: boolean} = {};
let pendingTypes: {name: string, symbol: ts.Symbol}[] = [];
for (const sourceFile of program.getSourceFiles()) {
if (sourceFile.fileName == fileName) {
ts.forEachChild(sourceFile, visit);
}
}
while (pendingTypes.length > 0) {
let pendingType = pendingTypes.shift();
printJsonType(pendingType.name, pendingType.symbol);
}
function visit(node: ts.Node) {
if (node.kind == ts.SyntaxKind.VariableStatement) {
(<ts.VariableStatement>node).declarationList.declarations.forEach(declaration => {
if (declaration.name.kind == ts.SyntaxKind.Identifier) {
let identifier = <ts.Identifier>declaration.name;
let symbol = checker.getSymbolAtLocation(identifier);
if (symbol) {
let t = checker.getTypeOfSymbolAtLocation(symbol, identifier);
if (t && t.symbol) {
pendingTypes.push({name: identifier.text, symbol: t.symbol});
}
}
}
});
}
}
function printJsonType(name: string, symbol: ts.Symbol) {
if (symbol.members) {
console.log(`export interface ${capitalize(name)} {`);
Object.keys(symbol.members).forEach(k => {
let member = symbol.members[k];
let typeName = null;
if (member.declarations[0]) {
let memberType = checker.getTypeOfSymbolAtLocation(member, member.declarations[0]);
if (memberType) {
typeName = getMemberTypeName(k, memberType);
}
}
if (!typeName) {
console.log(`// Sorry, could not get type name for ${k}!`);
} else {
console.log(` ${k}: ${typeName};`);
}
});
console.log(`}`);
}
}
function getMemberTypeName(memberName: string, memberType: ts.Type): string | null {
if (memberType.flags == ts.TypeFlags.String) {
return 'string';
} else if (memberType.flags == ts.TypeFlags.Number) {
return 'number';
} else if (0 !== (memberType.flags & ts.TypeFlags.Boolean)) {
return 'boolean';
} else if (memberType.symbol) {
if (memberType.symbol.name == 'Array' && (<ts.TypeReference>memberType).typeArguments) {
let elementType = (<ts.TypeReference>memberType).typeArguments[0];
if (elementType && elementType.symbol) {
let elementTypeName = capitalize(stripS(memberName));
if (!knownTypes[elementTypeName]) {
knownTypes[elementTypeName] = true;
pendingTypes.push({name: elementTypeName, symbol: elementType.symbol});
}
return `${elementTypeName}[]`;
}
} else if (memberType.symbol.name == '__object') {
let typeName = capitalize(memberName);
if (!knownTypes[typeName]) {
knownTypes[typeName] = true;
pendingTypes.push({name: typeName, symbol: memberType.symbol});
}
return typeName;
} else {
return null;
}
} else {
return null;
}
}
function capitalize(n: string) {
return n.charAt(0).toUpperCase() + n.slice(1);
}
function stripS(n: string) {
return n.endsWith('s') ? n.substring(0, n.length - 1) : n;
}
}
printInferredTypes([fileName], {
noEmitOnError: true, noImplicitAny: true,
target: ts.ScriptTarget.ES5, module: ts.ModuleKind.CommonJS
});
Found a npm package that converts a arbitrary JSON file without a schema into a TS interface:
https://www.npmjs.com/package/json-to-ts
The author also provided a VSCode plugin.
You can use an npm module instead of the web hosted solution:
https://www.npmjs.com/package/json-schema-to-typescript
If your JSON comes from an HTTP API and the API has an swagger code definition, you can generate a TypeScript client:
https://github.com/swagger-api/swagger-codegen#api-clients
If you json comes from a Java ot .Net backened, you can generate TypeScript from Java or C# classes:
http://type.litesolutions.net
https://github.com/raphaeljolivet/java2typescript
Using only sed and tsc
sed '1s#^#const foo = #' sample.json > sample.$$.ts
tsc sample.$$.ts --emitDeclarationOnly --declaration
Append const foo = to beginning of file
Using sed to replace (s) nothing (#^#) at the beginning of the first line (1) with const foo =
output to sample.$$.ts
the extension is the required to be .ts
$$ expands to the shells process id, handy for a temp file that is unlikely to overwrite stuff you care about
ask tsc to only emit a .d.ts typings file
this file has pretty much everything you want for the interface. You might need to replace a few strings and customize it in a way you want but most of the leg work is done
I want to be able to parse a string to an object that I can access using the dot notation e.g. myobject.property, instead of the array notation e.g. myobject['property']. The array notation works fine. Here's what I have so far.
I have some XML:
<level1 name="level1name">
<level2 type="level2Type">
<entry>level2entry</entry>
<entry>level2entry</entry>
</level2>
</level1>
Which converts to the JSON:
{
"level1": {
"name": "level1name",
"level2": {
"type": "level2Type",
"entry": [
"level2entry",
"level2entry"
]
}
}
}
I have the following Dart code:
Object jsonObject = JSON.parse("""{
"level1": {
"name": "level1name",
"level2": {
"type": "level2Type",
"entry": [
"level2entry",
"level2entry"
]
}
}
}
""");
print("my test 1 == ${jsonObject}");
print("my test 2 == ${jsonObject['level1']}");
print("my test 3 == ${jsonObject['level1']['name']}");
which produce the (desired) output:
my test 1 == {level1: {name: level1name, level2: {type: level2Type, entry: [level2entry, level2entry]}}}
my test 2 == {name: level1name, level2: {type: level2Type, entry: [level2entry, level2entry]}}
my test 3 == level1name
But when I try:
print("my test 1 == ${jsonObject.level1}");
I get the following:
Exception: NoSuchMethodException : method not found: 'get:level1'
Receiver: {level1: {name: level1name, level2: {type: level2Type, entry: [level2entry, level2entry]}}}
Arguments: []
Stack Trace: 0. Function: 'Object.noSuchMethod' url: 'bootstrap' line:717 col:3
Ideally, I want an object that I can access using the dot notation and without the compiler giving warning about Object not having property. I tried the following:
class MyJSONObject extends Object{
Level1 _level1;
Level1 get level1() => _level1;
set level1(Level1 s) => _level1 = s;
}
class Level1 {
String _name;
String get name() => _name;
set name(String s) => _name = s;
}
...
MyJSONObject jsonObject = JSON.parse("""{
"level1": {
"name": "level1name",
"level2": {
"type": "level2Type",
"entry": [
"level2entry",
"level2entry"
]
}
}
}
""");
...
print("my test 1 == ${jsonObject.level1.name}");
but instead of giving me 'level1name' as hoped, I get:
Exception: type 'LinkedHashMapImplementation<String, Dynamic>' is not a subtype of type 'MyJSONObject' of 'jsonObject'.
What am I doing wrong here? Is there any way to do what I'm trying? Thanks.
At the moment, JSON.parse only returns Lists (array), Maps, String, num, bool, and null
(api ref).
I suspect that until reflection makes it way into the language, it won't be able to re-construct objects based upon the keys found in json.
You could, however, create a constructor in your MyJsonObject which took a string, called JSON.parse internally, and assigned the various values.
Something like this works in the dart editor:
#import("dart:json");
class Level2 {
var type;
var entry;
}
class Level1 {
var name;
var level2;
}
class MyJSONObject {
Level1 level1;
MyJSONObject(jsonString) {
Map map = JSON.parse(jsonString);
this.level1 = new Level1();
level1.name = map['level1']['name'];
level1.level2 = new Level2();
level1.level2.type = map['level1']['level2']['type'];
//etc...
}
}
main() {
var obj = new MyJSONObject(json);
print(obj.level1.level2.type);
}
A non trivial version would needs some loops and possible recursion if you had deeper nested levels.
Update: I've hacked together a non-trivial version (inspired by the post below), it's up on github (also taking Seth's comments re the constructor):
Chris is completely right. I will only add that the JSON parser could be modified to return a little richer object (something like JsonMap instead of pure Map) that could allow jsonObj.property by implementing noSuchMethod. That would obviously perform worse than jsonObj['property'].