I am trying to upload JSON documents to Watson Discovery Service through the POST /documents API. The status code in the response is 202 which means that the document is accepted. But when I check in the Discovery Tooling, the error shown is:
"Error during indexing. The document contains a field type that is different than one with the same name in other documents."
What does this error mean? All the documents have the same fields & format. Some documents might have a few fields missing. I have also tried to upload this document using Discovery tooling, it still gives the same error. Are there any specific things that are checked during indexing JSON documents?
This usually happens when a document indexed has a different type in one or more fields than the document you want to index.
if you indexed the document below into your collection:
{
"id": "doc1",
"text": "Some text here"
}
And then tried to index the following document into the same collection:
{
"id": "doc2",
"text": {
"description": "Some description"
}
}
You'll get the error you mentioned above about the text field because one is a string and the other an object.
Getting an initial 202 is for the acceptance of the document itself. Validation of the compatibility of the final document with the collection happens later because the user can have configurations to normalize/delete fields that would cause conflicts with the existing collection.
"Missing" fields are fine for indexing.
Related
I'm facing some issues when generating a schema using a RFC connection to SAP while calling the function "BAPI_COMPANYCODE_GETLIST" which then creates a schema in my integration account using the content generated previously. However, after the schema is generated, when i access it all it has are the headers that are on the SAP table and none of its content. In this case the headers are COMP_NAME and COMP_CODE and it should return 122 rows but it does not and it also doesnt return any error so i cannot understand why it can retrieve the headers of the table but not its content.
I've tried enabling safe typing but after that the SAP connection doesnt work anymore, also tried calling different functions but the results are the same with different headers. Since this connector is recent i'm not able to find any solutions for this issue at this moment
The flow first receives an HTTP Request, afterwards it calls the BAPI function to generate the schema which is then used to create the schema in the integration account with the following properties:
{
"Content": "#{base64ToString(items('For_each')?['Content'])} ",
"ContentType": "application/xml",
"SchemaType": "Xml"
}
The schema is just that – the metadata describing the structure of an XML document. It is not the XML document itself.
The schema will contain two parts, the request message structure and the response message structure. You need to use the request message structure to form the BAPI get list, then can use the response message structure to parse the response. Either of the generic send message to SAP or the targeted Call BAPI actions of the SAP connector can be used to send the request message.
How to customise error message for invalid input?
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"username": {
"type": "string",
"pattern": "^[A-Za-z0-9-_.]+$",
"minLength": 3
},
"password": {
"type": "string",
"minLength": 8,
"pattern": "^(?=.*[a-z])(?=.*[A-Z])(?=.*\\d)[a-zA-Z\\d\\W]$"
}
},
"required": [
"username",
"password"
],
"errors": [
{
"property": "username",
"message": "min 3 characters, do not use spaces or special characters"
}
]
}
For example, if username input is not of required min length or doesn't satisfy regex pattern, display one custom message min 3 characters, do not use spaces or special characters
Custom error messages are not supported. However, there is some discussion going on to add a feature like this in the next version of JSON Schema.
Update 2021-01-26
JSON Schema never ended up supporting the customization of error messages in this way. The main problem with this is that appropriate error messaging is dependent on the audience and the context, so defining one message in this way is limiting. For example, a developer needs different feedback than an end user.
Instead, JSON Schema standardized the results that come back from a validation. This allows you to process the results to produce output that is appropriate for your audience. In theory libraries can be developed to produce error messaging for certain audiences. These would be decoupled from your validator library allowing you to more easily switch to another implementation in the future.
However, even the best error message producing libraries wouldn't solve the specific case presented in the original question. A library can't take a regular expression and produce a meaningful message. The good news is, JSON Schema provides an extension mechanism called vocabularies that you can use to create a custom keyword to annotate your schemas with the information that an output processor needs to produce a better error message. For example, the errors keyword in the original question would appear in the standard output results and can be used by an output processor as one of the ways it produces nice user facing error messages.
Unfortunately, no one has built one of these standard output processors yet, so you won't be able to pick one up off the shelf. It shouldn't be too hard to do, but you'd have to write it yourself. https://github.com/atlassian/better-ajv-errors is one of these output processor tools, but it uses ajv's proprietary output format rather than the standard format.
Both the standardized output format and JSON Schema Vocabularies are new in draft 2019-09 which has not seen big adoption so far. As time goes on, we expect to see more tools that make these kinds of customizations easy.
My current project, a restful API, validates a POST request to create a new user and multiple errors could occur (with HTTP status):
username not set (400 BadRequest)
username is taken (409 Conflict)
server can't establish db connection (500 Internal Server Error)
...
Should I immediatly send back a JSON response like this
{
"status": 400,
"Message": "No username is set"
}
if an error was detected or should is it better if I accumulate all errors like here:
{
"status": <HTTP STATUS CODE>,
"errors": [
{"message": "Username is not set."},
{"message": "Can't access the database."}
]
}
The last approach would not require multiple request to send a valid payload. But which status code should be used, if the username is not set (400 Bad Request) or the server can't access the database (500 Internal Server Error)?
I think if you foresee the need for multiple errors in one request, then the second JSON (with the multiple errors) is preferred. Another benefit of doing the multiple errors response is that as a user of your service, if i get back multiple errors, I can address them all at one shot, instead of addressing them one at a time as I get them.
After some research, the best (most standard) way to respond is a JSON structure of this form:
{
"error": {
"code": "400",
"message": "main error message here",
"target": "approx what the error came from",
"details": [
{
"code": "23-098a",
"message": "Disk drive has frozen up again. It needs to be replaced",
"target": "not sure what the target is"
}
],
"innererror": {
"trace": [ ... ],
"context": [ ... ]
}
}
}
The multiple errors you want to place would be individual elements in the "details" array. Using this structure you would still need some overall summary of the error, however the details would contain as many individual error messages as you want.
This is the format proposed by the OASIS data standard OASIS OData and seems to be the most standard option out there, however there does not seem to be high adoption rates of any standard at this point.
This also conforms to the JSON RPC 2.0 Spec as it requires that the error be an object in a "error" member, and that you have a code and message.
You can find the complete open source library that implements this at: Mendocino JSON Utilities. This library supports the JSON Objects as well as the exceptions.
The details are discussed in my blog post on Error Handling in JSON REST API
I was reading this Questions regarding REST
What exactly is RESTful programming?
While reading i get that the client is independent of server and client don't need to construct anything.
I want to know that when we are building forms like user registration . Then what is the REST way of doing it.
I mean when i do GET for /user/new then
Does the server has to send the complete FORM in html
Only send fields in JSON and form is constructed by client itself
But then again there will be many complexities, if i just send the fields, then what things like
Hidden fields
Default value for select boxes
what about some logic like this field can'r be greater than 30 etc
REST is, as you're already aware, a way of communicating between a client and a server. However, the issue here is what is being defined as the "client". Personally, I tend to consider that the browser itself is not in itself the client: instead, the client is written in JavaScript, and the browser is merely a conduit to executing it.
Say for the sake of argument that you wish to view the details of user '1414'. The browser would be directed to the following location:
/UserDetails.html#1414
This would load the static file ViewUser.html, containing all the form fields that may be necessary, as well as (via a <script> tag) your JavaScript client. The client would load, look at the URL and make a RESTful call to:
GET /services/Users/1414
which would send back JSON data relating to that user. When the user then hits "save", the client would then make the following call:
PUT /services/Users/1414
to store the data.
In your example, you wanted to know how this would work with a new user. The URL that the browser would be directed to would be:
/UserDetails.html#0
(or #new, or just # - just something to tell the JavaScript that this is a new client. This isn't a RESTful URL so the precise details are irrelevant).
The browser would again load the static file ViewUser.html and your JavaScript client, but this time no GET would be made on the Users service - there is no user to download. In addition, when the user is saved, this time the call would be:
POST /services/Users/
with the service ideally returning a 302 to /services/Users/1541 - the location of the object created. Note that as this is handled in the client not the browser, no actual redirection occurs.
"Forms" for hypermedia APIs could be rendered in a "forms aware" media type like for instance Mason (https://github.com/JornWildt/Mason), Hydra (http://www.markus-lanthaler.com/hydra/) or Sirene (https://github.com/kevinswiber/siren). In Mason (which is my project) you could have a "create user" action like this:
{
"#actions": {
"create-user": {
"type": "json",
"href": "... URL to resource accepting the POST ...",
"method": "POST",
"title": "Create new user",
"schemaUrl": "... Optional URL to JSON schema definition for input ..."
"template": {
"Windows Domain": "acme"
}
}
}
}
The client can GET a resource that include the above action, find it be the name "create-user" and in this way be told which method to use, where to apply it, how the payload should be formated (in this case its JSON as described by an external schema definition) and some default values (the "template" object).
If you need more complex descriptions (like selection lists and validation rules as you mention) then you are on your own and will have to encoded that information in your own data - or use HTML or XForms.
There are multiple ways to do what you want.
You can use GET for /user/new along with a create-form link relation to get a single link. This can in plain HTML or HTML fragment, or a schema description, practically anything you want (the result will be less reusable than the other solutions).
You can use a standard MIME type which supports form descriptions. For example HAL with a form extension or collection+json.
You can use an RDF format, like JSON-LD with a proper vocab like Hydra.
Suppose I have a few web forms to implement. The forms contain standard greetings, validation messages (e.g. "missing name", "email address is invalid"), errors (e.g. "temporary processing error"), etc.
Does it make sense to factor out all these text messages from the HTML and store it in an external text file so that non-technical people might edit the text ?
They say it is easier to edit text files instead of HTML. On the other hand I am afraid it would complicate the solution. What are the best practices in that field?
It depends from case. If texts are often edited by nontechnical people, it may make sense to move text into a separate file with simple structure. Otherwise, it indeed could complicate things.
Typically, server-side template engine is used to build pages from multiple different resources (such resources are, e.g., HTML-template files, database, configuration files, etc.). What type of resource and format of it to use is up to you and depends on situation. For example, you could store your error texts in JSON-format files like this:
{
"name" : {
"minlength" : {
"value": 2,
"error": "Name field must contain at least 2 characters"
},
"maxlength" : {
"value": 255,
"error": "Name field must contain not more than 255"
}
},
"email" : {
"pattern" : {
"value": "some_regexp_for_email_validation",
"error": "Please input a correct e-mail address"
}
}
}
In PHP in particular, JSON format can be read with json_decode() method.
An alternative to JSON is XML (though it's typically harder to use).
By the way, it may make sense to provide a web interface to edit form error rules and texts for nontechnical people. Then implementation details would be hidden from people who shouldn't know about them. So you could use whatever you want as for technical part of this while editors would see just a usable GUI with text fields.
You also may be interested in using ready server-side data-validation solutions like Zend Validator.
I'm using an java webapp which uses keys that mapped to strings in *.properties file.
I noticed that it's more difficult to support such code in cases you're searching "where's that field label "some cool field":
First you have to find key (ok, you get that key for that string is "submit.button.text"), and then you'll have to find where's key is actully used in your code.