I am trying to write a json schema including the feature to check if a path is valid and exist.
For example I want to validate this json:
{
"paths": ["/path/to_check", "../path/not/valid", "../../path/exists"]
}
My current schema is:
{
"type": "object",
"properties": {
"paths": {
"type": "array",
"items": {
"type": "string"
}
}
}
}
Is there a way to indicate that items must contain valid/existing paths?
You can use regex, but there's no way to determine if a path is a real path according to a file system. JSON Schema works with JSON data... that's all, nothing more. It has no notion of a file system.
I've been googling for the same thing and was coming to the same conclusion that #Relequestual posted (who is the authority on such things).
What may be of interest is that the pydantic library extends their JSON Schema with a bunch of extensions for complex string sub-types, including file-path, directory-path and path (and many, many more).
This could be useful either directly, or to adopt as a quasi-standard for a custom implementation.
Related
I want to split my paths (which are quite many) more easily into their own files.
Let's say I've got two major paths /user and /anotherPath with several subpaths. Now I've got an OpenApi spec file, whose paths are being referenced to an index file which holds references to each paths. Defining EVERY path with its' reference works, but is clumsy to write.
I want something like this:
openapi.json
{
...
"paths": {
"$ref": "paths/index.json"
}
...
}
paths/index.json
{
"/user": { // and everything that comes after user, e.g. /user/{userId}
"$ref": "./user-path.json"
},
"/anotherPath": { // and everything that comes after anotherPath, e.g. /anotherPath/{id}
"$ref": "./anotherPath-path.json"
}
}
paths/user-path.json
{
"/user": {
"get": {...}
},
"/user/{userId}": {
"get": {...}
}
}
paths/anotherPath-path.json
{
"/anotherPath": {
"get": {...}
},
"/anotherPath/{id}": {
"get": {...}
}
}
This way, whenever I add another path to /user or /anotherPath, I can simply edit their respective path file, e.g. paths/user-path.json.
EDIT1: Apparently, this topic is being discussed already.. For anyone interested: https://github.com/OAI/OpenAPI-Specification/issues/417 . By the way, I know that a $ref is not valid for the paths Object, but once figuring out how to properly split, this may not be necessary anymore.
OpenAPI does not have a concept of sub-paths / nested paths, each path is an individual entity. The paths keyword itself does not support $ref, only individual paths can be referenced.
Given your user-path.json and anotherPath-path.json files, the correct way to reference path definitions is as follows:
{
...
"paths": {
"/user": {
"$ref": "paths/user-path.json#/~1user" // ~1user is /user escaped according to JSON Pointer and JSON Reference rules
},
"/user/{id}": {
"$ref": "paths/user-path.json#/~1user~1%7Bid%7D" // ~1user~1%7Bid%7D is /user/{id} escaped
},
"/anotherPath": {
"$ref": "paths/anotherPath-path.json#/~1anotherPath" // ~1anotherPath is /anotherPath escaped
},
"/anotherPath/{id}": {
"$ref": "paths/anotherPath-path.json#/~1anotherPath~1%7Bid%7D" // ~1anotherPath~1%7Bid%7D is /anotherPath/{id} escaped
}
}
...
}
YAML version:
paths:
/user:
$ref: "paths/user-path.json#/~1user"
/user/{id}:
$ref: "paths/user-path.json#/~1user~1%7Bid%7D"
/anotherPath:
$ref: "paths/anotherPath-path.json#/~1anotherPath"
/anotherPath/{id}:
$ref: "paths/anotherPath-path.json#/~1anotherPath~1%7Bid%7D"
If you want to use $ref in arbitrary places (other than where OAS allows $refs), you'll have to pre-process your definition using a parser/tool that can resolve arbitrary $refs; this will give you a valid OpenAPI file that can be used with OpenAPI-compliant tools. One such pre-processing tool is json-refs, you can find an example of pre-processing here.
Is there a way to split a swaggerfile / OpenAPI specs file, either JSON or YAML, encoding every $ref into a separate file? Because I found a lot of solutions to achieve the opposite (multiple files -> single file), but none for this.
What I'd like to achieve is the following:
I have a huge JSON swaggerfile that contains internal $refs.
I'd like to have a single file for each and every object or path definition, and, in the root file, references (local or absolute) to these files. This way I can edit the root file to easily obtain a minimal subset of the paths and objects that I need.
{
"in": "inputField",
"required": true,
"schema": {
"$ref": "#/components/schemas/MyObject"
},
"components": {
"schemas": {
"MyObject": {
"type": "object",
"properties": {
"value": {
"type": "string"
}
}
}
}
},
"____comment": "I want MyObject definition in MyObject.json file and the $ref to that file"
}
Yeah, its possible, you must only use regex, and detect depedencies etc.
regex example from my project
new Regex("\"#/definitions/(.*)\"");
new Regex("#\\/definitions\\/(.*?)\\\"");
new Regex("\"" + key + "\": ");
etc.
You must replace elements with dependencies, and save to file. I do sth like that to recive jschema from model. Your case,is a little diffrent, but smilar.
Trying to figuring out how to deserialize this kind of json in talend components :
{
"ryan#toofr.com": {
"confidence":119,"email":"ryan#toofr.com","default":20
},
"rbuckley#toofr.com": {
"confidence":20,"email":"rbuckley#toofr.com","default":15
},
"ryan.buckley#toofr.com": {
"confidence":18,"email":"ryan.buckley#toofr.com","default":16
},
"ryanbuckley#toofr.com": {
"confidence":17,"email":"ryanbuckley#toofr.com","default":17
},
"ryan_buckley#toofr.com": {
"confidence":16,"email":"ryan_buckley#toofr.com","default":18
},
"ryan-buckley#toofr.com": {
"confidence":15,"email":"ryan-buckley#toofr.com","default":19
},
"ryanb#toofr.com": {
"confidence":14,"email":"ryanb#toofr.com","default":14
},
"buckley#toofr.com": {
"confidence":13,"email":"buckley#toofr.com","default":13
}
}
This JSON comes from the Toofr API where documentation can be found here .
Here the actual sitation :
For each line retreived in the database, I call the API and I got this (the first name, the last name and the company change everytime.
Does anyone know how to modify the tExtractJSONField (or use smthing else) to show the results in tLogRow (for each line in the database) ?
Thank you in advance !
EDIT 1:
Here's my tExtractJSONfields :
When using tExtractJSONFields with XPath, you need
1) a valid XPath loop point
2) valid XPath mapping to your structure relative to the loop path
Also, when using XPath with Talend, every value needs a key. The key cannot change if you want to loop over it. Meaning this is invalid:
{
"ryan#toofr.com": {
"confidence":119,"email":"ryan#toofr.com","default":20
},
"rbuckley#toofr.com": {
"confidence":20,"email":"rbuckley#toofr.com","default":15
},
but this structure would be valid:
{
"contact": {
"confidence":119,"email":"ryan#toofr.com","default":20
},
"contact": {
"confidence":20,"email":"rbuckley#toofr.com","default":15
},
So with the correct data the loop point might be /contact.
Then the mapping for Confidence would be confidence (the name from the JSON), the mapping for Email would be email and vice versa for default.
EDIT
JSONPath has a few disadvantages, one of them being you cannot go higher up in the hierarchy. You can try finding out the correct query with jsonpath.com
The loop expression could be $.*. I am not sure if that will satisfy your need, though - it has been a while since I've been using JSONPath in Talend because of the downsides.
I have been ingesting some complex json structures and did this via minimal json libraries, and tjava components within talend.
Neo4j Version: 3.2.2
Operating System: Ubuntu 16.04
I use getDegree() function in mapping.json file, but the return would always be null; I'm using the dataset neo4j tutorial Movie/Actor dataset.
Output from elasticsearch request
mapping.json
{
"defaults": {
"key_property": "uuid",
"nodes_index": "default-index-node",
"relationships_index": "default-index-relationship",
"include_remaining_properties": true
},
"node_mappings": [
{
"condition": "hasLabel('Person')",
"type": "getLabels()",
"properties": {
"getDegree": "getDegree()",
"getDegree(type)": "getDegree('ACTED_IN')",
"getDegree(direction)": "getGegree('OUTGOING')",
"getDegree('type', 'direction')": "getDegree('ACTED_IN', 'OUTGOING')",
"getDegree-degree": "degree"
}
}
],
"relationship_mappings": [
{
"condition": "allRelationships()",
"type": "type",
}
]
}
Also if I use isOutgoing(), isIncoming(), otherNode function in relationship_mappings properties part, elasticsearch would never load the relationship data from neo4j. I think I probably have some misunderstanding of this sentence only when one of the participating nodes "looking" at the relationship is provided on this page https://github.com/graphaware/neo4j-framework/tree/master/common#inclusion-policies
mapping.json
{
"defaults": {
"key_property": "uuid",
"nodes_index": "default-index-node",
"relationships_index": "default-index-relationship",
"include_remaining_properties": true
},
"node_mappings": [
{
"condition": "allNodes()",
"type": "getLabels()"
}
],
"relationship_mappings": [
{
"condition": "allRelationships()",
"type": "type",
"properties": {
"isOutgoing": "isOutgoing()",
"isIncomming": "isIncomming()",
"otherNode": "otherNode"
}
}
]
}
BTW, is there any page that list all of the functions that we can use in mapping.json? I know two of them
github.com/graphaware/neo4j-framework/tree/master/common#inclusion-policies
github.com/graphaware/neo4j-to-elasticsearch/blob/master/docs/json-mapper.md
but it seems there are more, since I can use getType(), which hasn't been listed in any of the above pages.
Please let me know if I can provide any further help to solve the problem
Thanks!
The getDegree() function is not available to use, in contrary to getType(). I will explain why :
When the mapper (the part responsible to create a node or relationship representation as ES document ) is doing its job, it receive a DetachedGraphObject being a detached node or relationship.
The meaning of detached is that it is happening outside of a transaction and thus query operations are not available against the database anymore. The getType() is available because it is part of the relationship metadata and it is cheap, however if we would want to do the same for getDegree() this can be seriously more costly during the DetachedObject creation (which happen in a tx) depending on the number of different types etc.
This is however something we are working on, by externalising the mapper in a standalone java application coupled with a broker like kafa, rabbit,.. between neo and this app. We would not, however offer the possibilty to requery the graph in the current version of the module as it can have serious performance impacts if the user is not very careful.
As last, the only suggestion I can give you is to keep a property on your node with the updates of degrees you need to replicate to ES.
UPDATE
Regarding this part of the documentation :
For Relationships only when one of the participating nodes "looking" at the relationship is provided:
This is used only when not using the json definition, so you can use one or the other. the json definition has been added later as addition and both cannot be used together.
For answering this part, it means that the nodes of the incoming or outgoing side, depending on the definition, should be included in the inclusion policy for nodes, like hasLabel('Employee') || hasProperty('form') || getProperty('age', 0) > 20 . If you have an allNodes policy then it is fine.
I'm having trouble properly formatting one particular soap parameter using the node-soap module for node.js as a client, to a 3rd-party SOAP service.
The client.describe() for this method says this particular input should be in the shape of:
params: { 'param[]': {} }
I have tried a bunch of different JSON notations to try to fit my data to that shape.
Examples of formats that do NOT work:
"params": { "param": [ {"myParameterName": "myParameterValue"} ] }
"params": [ "param": { "name": "myParameterName", "_": "myParameterValue"} ]
"params": { "param" : [ {"name": "myParameterName", "_": "myParameterValue"} ] }
"params": { "param[]": {"myParameterName": "myParameterValue" } }
"params": { "param[myParameterName]": {"_": "myParameterValue" } }
I must be overlooking something, and I suspect I'm going to feel like Captain Obvious when some nice person points out what I'm doing wrong.
Here is what DOES work, using other soap clients, and how they handle the "named parameter with a value"
soapUI for this method successfully accepts this particular input via XML in the shape of:
<ns:params>
<ns:param name="myParameterName">myParameterValue</ns:param>
</ns:params>
Also, using PHP, I can successfully make the call by creating a stdClass of arrays like so:
$parms = new stdClass;
$parms->param = array(
array(
"name"=>"myParameterName","_"=>"myParameterValue"
)
);
and then eventually passing
'params' => $parms
to the PHP soap client
Many thanks!
To get a better look at what XML was being generated by node-soap, I added a console.log(message) statement to the node_modules/soap/lib/client.js after the object-to-XML encoding. I then began experimenting with various JSON structures to figure out empirically how they were mapping to XML structures.
I found a JSON structure for node-soap to generate the XML in my 3rd-party's required named-parameter-with-value format. I was completely unaware of the "$value" special keyword. Looks like this may have been added in the 0.4.6 release from mid-June 2014. See the change history
"params": [
{
"param": {
"attributes": {
"name": "myParameterName"
},
$value: "myParameterValue"
}
}
]
(note the outer array, which gives me the luxury of specifying multiple "param" entries, which is sometimes needed by this particular 3rd-party API)
generates this XML:
<tns:params>
<tns:param name="myParameterName">myParameterValue</tns:param>
</tns:params>
which perfectly matches the structure in soapUI (which I already knew worked) of:
<ns:params>
<ns:param name="myParameterName">myParameterValue</ns:param>
</ns:params>