I am using log4j2 with a JSON properties file I am naming log4j2.json and attempting to add a kafka appender. There are plenty of examples using the older formats or xml, but struggling to get the correct format in JSON. This is most likely a silly question, but I have been struggling to get this to work and can find no examples anywhere. I am tempted to abandoned configuring log4j2 in json and moving to XML, but I feel this should be rather simple.
Here is a sample kafka appender defined in XML.
'<Appenders>
<Kafka name="Kafka" topic="devglan-test">
<PatternLayout pattern="%date %message"/>
<Property name="bootstrap.servers">localhost:9092</Property>
</Kafka>
<Async name="Async">
<AppenderRef ref="Kafka"/>
</Async>
</Appenders>
How does the
'localhost:9092
entry map to JSON.
I have tried a number of styles such as the following:
'"Kafka": {
"name":"Kafka",
"topic":"FEEDPROCESSING_Dev",
"PatternLayout": {
"pattern": "%date %message"
},
"bootstrap.servers": [{"name":"localhost:9092"}]}
}
That did not work. I also tried..
'"Kafka": {
"name":"Kafka",
"topic":"FEEDPROCESSING_SYNC_Dev",
"PatternLayout": {
"pattern": "%date %message"
},
"Property": {"bootstrap.servers":"localhost:9092"}}
}
I am getting errors like:
Property contains an invalid element or attribute "bootstrap.servers"
THis configuration gives me the error "Unable to locate plugin type for bootstrap.servers":
'"Property": {"bootstrap.servers":[{"name":"localhost:9092"}]}}},
I know I have the correct maven dependencies..
What is the correct way to configure a Kafka appender, particular the bootstrap server property,using JSON and log4j2?
This is the correct format I think:
"Kafka": {
"name": "Kafka",
"topic": "FEEDPROCESSING_Dev",
"PatternLayout": {
"pattern": "%date %message"
},
"Property": {
"name": "bootstrap.servers",
"value": "localhost:9092"
}
}
Related
I have a JSON file that is going to contain a number of different lists per client that I am deploying for. These lists are going to serve as container overrides for an ECS task that my Lambda function will be invoking. The JSON config file would look something like this:
{
"clientName": {
"environment": [
{
"name": "name1",
"value": "value1"
}
]
}
}
And my serverless.yml would look something like this:
environment:
CONTAINER_ENVIRONMENT: ${file(serverlessConfig.json):${env:CLIENT_NAME}.environment}
Which results in the following error:
Could not resolve "CONTAINER_ENVIRONMENT" environment variable: Unsupported environment variable format:
[
{
"name": "name1",
"value": "value1"
}
]
I've tried using CloudFormation intrinsic functions such as Fn::Join and Fn::ToJsonString. These both threw an error when trying to run locally using sls invoke local (the errors were the same as the above). After some digging it seems that these functions aren't compatible with the serverless environment property.
The only thing that has worked so far is storing the list as a string in a .env file, but that's not really ideal since these configs could take a number of different environment objects.
Is there any way to get this to work with the setup that I have going?
I am trying to write a json schema including the feature to check if a path is valid and exist.
For example I want to validate this json:
{
"paths": ["/path/to_check", "../path/not/valid", "../../path/exists"]
}
My current schema is:
{
"type": "object",
"properties": {
"paths": {
"type": "array",
"items": {
"type": "string"
}
}
}
}
Is there a way to indicate that items must contain valid/existing paths?
You can use regex, but there's no way to determine if a path is a real path according to a file system. JSON Schema works with JSON data... that's all, nothing more. It has no notion of a file system.
I've been googling for the same thing and was coming to the same conclusion that #Relequestual posted (who is the authority on such things).
What may be of interest is that the pydantic library extends their JSON Schema with a bunch of extensions for complex string sub-types, including file-path, directory-path and path (and many, many more).
This could be useful either directly, or to adopt as a quasi-standard for a custom implementation.
I was using a Mule application to get data from another system which is invoked by an HTTP endpoint, used java with jersey api to get the rest component.
Sample input data is given below, here keys and values are not fixed, it may vary based on user request. Array size will increase may be 2 to n entries. It is working fine with Mule and Java rest based component.
Input JSON data:
[
{
"Company": "BEG1",
"Account": "10011",
"Deptid": "111",
"Location": "SM1",
"Transaction Date": "2014-07-15",
"Description": "Invoice1",
"Debit": 0,
"Credit": 13.46,
"Invoice Nbr": "16824321"
},
{
"Company": "BEG92",
"Account": "10092",
"Deptid": "222",
"Location": "SL2",
"Transaction Date": "2014-07-19",
"Description": "Invoice End2",
"Debit": 13.46,
"Credit": 0,
"Invoice Nbr": "168243292"
}
]
Planning to migrate to APIkit with RAML: how can I make a RAML template for above case? Since keys are dynamic, this doesn't seem straightforward.
2) With same approach for GET, I will get the data for my get request, I am not sure what is the key and its corresponding values, only I was doing is get the data, parse it and send it to the user. How do I create RAML template on this situation.Will mule APIkit with RAML will work here?
My existing code:
#POST
#Path("/post")
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
public List<GLExport> postOperation(#Payload String content) throws ParseException {
JSONParser jsonParser = new JSONParser();
Object jsonObjectInstance =null;
jsonObjectInstance = jsonParser.parse(new StringReader(content));
...
return glExportList;
<http:inbound-endpoint exchange-pattern="request-response" host="${hostname}" port="${glport}" path="QBJournalExport/QBGLRest" doc:name="HTTP"/>
<jersey:resources doc:name="REST">
<component class="com.qb.rest.GLExportService"/>
</jersey:resources>
It seems your question is more about JSON Schema than RAML. In essence, you are asking how to support unknown fields in a JSON Schema.
The answer is by adding:
"additionalProperties": true
to your JSON Schema object definition.
I strongly suggest that if there is set of fields are known, you declare them explicitly in your schema: users of your API will thank you for that.
Also, if some of these fields are guaranteed to be present, mark them as required as well.
APIkit should have no trouble dealing with additional properties.
Here is a JSON instance showing the start-time and end-time for a meeting:
{
"start time": "2015-02-19T08:00:00Z",
"end time": "2015-02-19T09:00:00Z"
}
I can specify the structure of that instance using JSON Schema: the instance must contain an object with a "start time" property and an "end time" property and each property must be a date-time formatted string. See below for the JSON schema. But what I cannot specify is this: the meeting must start before it ends. That is, the value of "start time" must be less than the value of "end time". Some people call this data dependency a co-constraint. In the XML world there is a wonderful, simple technology for expressing co-constraints: Schematron. I am wondering if there is an equivalent technology in the JSON world? What would you use to declaratively describe the relationship between the value of "start time" and "end time"? (Note: writing code in some programming language is not what I mean by "declaratively describe the relationships". I am seeking a declarative means to describe the data dependencies that are present in JSON documents, not procedural code.)
{
"$schema": "http://json-schema.org/draft-04/schema#",
"definitions": {
"meeting": {
"type": "object",
"properties": {
"start time": { "type": "string", "format": "date-time"},
"end time": { "type": "string", "format": "date-time"}
},
"required": [ "start time", "end time" ],
"additionalProperties": false
}
},
"$ref": "#/definitions/meeting"
}
Yes.There is a JSON Semantic Validator based on Schematron available at:
https://www.npmjs.com/package/jsontron
It implements 'schema', 'phase', 'rule', 'assert' and reporting features of Schematron.
Here is when the original example of start time and end time was run through the validator:
good_time.json file contents:
{
"starttime": "2015-02-19T08:00:00Z",
"endtime": "2015-02-19T09:00:00Z"
}
bad_time.json file contents:
{
"starttime": "2015-02-19T09:00:00Z",
"endtime": "2015-02-19T08:00:00Z"
}
Schematron Rules file meeting-times-rules.json snippet:
"rule":[
{
"context": "$",
"assert":[
{
"id":"start_stop_meeting_chec",
"test":"jp.query(contextNode, '$..starttime') < jp.query(contextNode, '$..endtime')",
"message": "Meeting cannot end before it starts"
}
]
}
]
When ran with correct example:
$jsontron\bin>node JSONValidator -i ./good_time.json -r ./meeting-times-rules.json
The output was:
Starting Semantic Validation .........
Parsing Pattern: Meetingtimings
1 Pattern(s) Requested. 1 Pattern(s) Processed. 0 Pattern(s) Ignored.
**** THIS INSTANCE IS SEMANTICALLY VALID ****
Completed Semantic Validation .........
When ran with bad data example. The output was:
$jsontron\bin>node JSONValidator -i ./bad_time.json -r ./meeting-times-rules.json
Starting Semantic Validation .........
Parsing Pattern: Meetingtimings
1 Pattern(s) Requested. 1 Pattern(s) Processed. 0 Pattern(s) Ignored.
**** THIS INSTANCE CONTAINS SEMANTIC VALIDATION ISSUES. PLEASE SEE FULL REPORT BY ENABLING DEBUG WITH -d OPTION ****
Completed Semantic Validation .........
The message with debug options was:
...validation failed...
message: 'Meeting cannot end before it starts'
Sadly, the answer is no. JSON Schema allows you to validate the structure, and permitted values, but there are no mechanisms for validating sets of values, a'la Schematron.
The simplest way to solve this is to have another script in the pipeline which runs these kinds of checks.
There is an implementation in Oxygen JSON Editor that allows you to validate JSON documents against Schematron.
https://www.oxygenxml.com/doc/versions/22.0/ug-editor/topics/json-validating-documents-against-schema.html
The Schematron rules are expressed using XPath expressions, and the problems are reported in the JSON documents.
<!-- The 'genre' property should be none but one of the items declared in 'literatureGenres' property -->
<sch:rule context="genre">
<sch:let name="genre" value="text()"/>
<sch:let name="literatureGenres" value="//literatureGenres/text()"/>
<sch:assert test="normalize-space($genre) = $literatureGenres">
Wrong genre: '<sch:value-of select="$genre"/>'. See the 'literatureGenres' property for the permitted ones.
</sch:assert>
</sch:rule>
https://www.slideshare.net/nottavy/schematron-for-nonxml-languages
The json-schema.org website lists quite a few implementations.
I'm having trouble properly formatting one particular soap parameter using the node-soap module for node.js as a client, to a 3rd-party SOAP service.
The client.describe() for this method says this particular input should be in the shape of:
params: { 'param[]': {} }
I have tried a bunch of different JSON notations to try to fit my data to that shape.
Examples of formats that do NOT work:
"params": { "param": [ {"myParameterName": "myParameterValue"} ] }
"params": [ "param": { "name": "myParameterName", "_": "myParameterValue"} ]
"params": { "param" : [ {"name": "myParameterName", "_": "myParameterValue"} ] }
"params": { "param[]": {"myParameterName": "myParameterValue" } }
"params": { "param[myParameterName]": {"_": "myParameterValue" } }
I must be overlooking something, and I suspect I'm going to feel like Captain Obvious when some nice person points out what I'm doing wrong.
Here is what DOES work, using other soap clients, and how they handle the "named parameter with a value"
soapUI for this method successfully accepts this particular input via XML in the shape of:
<ns:params>
<ns:param name="myParameterName">myParameterValue</ns:param>
</ns:params>
Also, using PHP, I can successfully make the call by creating a stdClass of arrays like so:
$parms = new stdClass;
$parms->param = array(
array(
"name"=>"myParameterName","_"=>"myParameterValue"
)
);
and then eventually passing
'params' => $parms
to the PHP soap client
Many thanks!
To get a better look at what XML was being generated by node-soap, I added a console.log(message) statement to the node_modules/soap/lib/client.js after the object-to-XML encoding. I then began experimenting with various JSON structures to figure out empirically how they were mapping to XML structures.
I found a JSON structure for node-soap to generate the XML in my 3rd-party's required named-parameter-with-value format. I was completely unaware of the "$value" special keyword. Looks like this may have been added in the 0.4.6 release from mid-June 2014. See the change history
"params": [
{
"param": {
"attributes": {
"name": "myParameterName"
},
$value: "myParameterValue"
}
}
]
(note the outer array, which gives me the luxury of specifying multiple "param" entries, which is sometimes needed by this particular 3rd-party API)
generates this XML:
<tns:params>
<tns:param name="myParameterName">myParameterValue</tns:param>
</tns:params>
which perfectly matches the structure in soapUI (which I already knew worked) of:
<ns:params>
<ns:param name="myParameterName">myParameterValue</ns:param>
</ns:params>