Error adding docs to CouchDB - json

My local PouchDB docs won't replicate to my remote CouchDB.
The sync is happening, because the browser downloaded my design schema, so it's not a permissions issue. I think that my design schema doesn't match my documents, but I struggled to find the correct way to write the schema.
Schema
{
"_id": "_design/schema",
"_rev": "4-3d9a49ebffbbd6b7b146240879baa7e4",
"validate_doc_update": "function(newDoc, oldDoc, userCtx, secObj){ if(userCtx.roles[0] !== 'admin'){throw({forbidden: 'operation forbidden'})} }",
"views": {
"by_module": {
"map": "function(doc){ if(doc.type == 'note'){emit(doc.note);} }"
}
},
"schema": {
"title": "Contact details",
"description": "A document containing a person's contact details.",
"type": "object",
"required": [
"name",
"level"
],
"properties": {
"_id": {
"type": "string"
},
"_rev": {
"type": "string"
},
"application_access": {
"type": "string"
},
"home": {
"type": "string"
},
"home_email": {
"type": "string"
},
"jobtitle": {
"type": "string"
},
"level": {
"type": "string"
},
"mobile1": {
"type": "string"
},
"mobile2": {
"type": "string"
},
"modified": {
"type": "number"
},
"name": {
"type": "string"
},
"work": {
"type": "string"
},
"work_email": {
"type": "string"
},
"_doc_id_rev": {
"type": "string"
}
}
}
}
Doc
{
"_id": "fcb52b3072e2038647b328c0a700147f",
"_rev": "1518449239461",
"application_access": "User",
"home": "",
"home_email": "",
"jobtitle": "Exemplar",
"level": "Bronze",
"mobile1": "0987654321",
"mobile2": "",
"modified": 1518449239461,
"name": "Zachary Zumbeispiel",
"work": "",
"work_email": "",
"_doc_id_rev": "1518449239460::1-ffd3c056614845ada4a68de4793710ac"
}
So the question is, does my doc conform to my schema? Or is my schema wrong?
The "schema" was on the CouchDB instance, and the doc was in PouchDB.

They are replicating now, because I removed the "document schema" and so it can sync & replicate.
So the problem was that I had read somewhere that I needed to add a schema to CouchDB, so I created what I thought was a schema, and evidently the structure of the docs to be synced has to be the same, so the sync did not work because the structure of the docs did not match.
Hence #Flimzy 's explanation that the whole notion of a schema was bobbins meant that I removed the "schema", and voila, the PouchDB and CouchDB can now sync - and problem solved.

Related

JsonSchema definition path and subschema re-use

Lets say I have two schemas defined as follows -
ADDRESS_CLASS_SCHEMA_DEFINITION = {
"title": "Address",
"type": "object",
"properties": {
"country_code": {
"$ref": "#/definitions/CountryCode"
},
"city_code": {
"title": "City Code",
"type": "string"
},
"zipcode": {
"title": "Zipcode",
"type": "string"
},
"address_str": {
"title": "Address Str",
"type": "string"
}
},
"required": [
"country_code",
"city_code",
"zipcode"
],
"definitions": {
"CountryCode": {
"title": "CountryCode",
"description": "An enumeration.",
"enum": [
"CA",
"USA",
"UK"
],
"type": "string"
}
}
}
EMPLOYEE_CLASS_SCHEMA_DEFINITION = {
"title": "Employee",
"type": "object",
"properties": {
"id": {
"title": "Id",
"type": "integer"
},
"name": {
"title": "Name",
"type": "string"
},
"email": {
"title": "Email",
"type": "string"
},
"telephone": {
"title": "Telephone",
"type": "string"
},
"address": {
"$ref": "#/definitions/Address"
}
},
"required": [
"id",
"name",
"email"
],
"definitions": {
"Address": ADDRESS_CLASS_SCHEMA_DEFINITION
}
}
I'm trying to re-use sub-schema definitions by defining a constant and referencing them individually in definitions (for example address-schema is referenced through constant in employee-schema definition). This approach works for individual schemas, however there seems to be a json-pointer path issue for Employee schema - #/definitions/CountryCode wouldn't resolve in Employee schema. I was assuming that #/definitions/CountryCode would be a relative path on Address schema as its scope is defined on a sub-schema, but my understanding seems wrong. I can make it work by flattening out like below, however I donot want to take this route -
{
"title": "Employee",
"type": "object",
"properties": {
"id": {
"title": "Id",
"type": "integer"
},
"name": {
"title": "Name",
"type": "string"
},
"email": {
"title": "Email",
"type": "string"
},
"telephone": {
"title": "Telephone",
"type": "string"
},
"address": {
"$ref": "#/definitions/Address"
}
},
"required": [
"id",
"name",
"email"
],
"definitions": {
"CountryCode": {
"title": "CountryCode",
"description": "An enumeration.",
"enum": [
"CA",
"USA",
"UK"
],
"type": "string"
},
"Address": {
"title": "Address",
"type": "object",
"properties": {
"country_code": {
"$ref": "#/definitions/CountryCode"
},
"city_code": {
"title": "City Code",
"type": "string"
},
"zipcode": {
"title": "Zipcode",
"type": "string"
},
"address_str": {
"title": "Address Str",
"type": "string"
}
},
"required": [
"country_code",
"city_code",
"zipcode"
]
}
}
}
I'm wondering how to fix this, I've briefly looked into jsonschema-bundling and using $id but from best practices it seems like the general recommendation is to use $id when dealing with URI's alone. Would like to know about best practices and how to fix this problem, would also appreciate if someone can point me on how to use $id correctly (for example, constant based approach seems to work when I provide identifiers like $id: Address, $id: Employee). Thanks in advance.
JSON Schema implementations work in JSON land. When you combine your schemas in your example above, presumably in javascript/node.js, by the time it gets to the JSON Schema implementation for validation execution, any knowledge that there were separate schemas is lost. (It's generally not considered that this approach is the best approach.)
The EASY fix here SHOULD be just to define $id in each of the roots of your schemas. These should be a fully qualfied URI. It doesn't really matter what they are at this point. They could be https://example.com/a and https://example.com/b. Then, in the primary schema, you can do $ref: https://example.com/b.
Implementations should provide you with a way to load in your other/non-primary schemas so the $id values can be stored in an index. Using $id in your other schema with a fully qualified URI will signify a "resource boundary".
https://json-schema.hyperjump.io is the only web playground to support multiple files/schemas/"Schema Resources", so you can test this out there to confirm your expectations.
Not all implementations make it easy or even provide a means to import your other schemas, but they should.
If you have follow up questions, feel free to leave a comment, or join the JSON Schema slack server if it would be off-topic for StackOverflow.

How to a build a JSON file from a CSV using an existing JSON schema format?

I have a JSON schema and a CSV file. The CSV file has 2,511 rows and one header row (2,512 rows total). Each row has 43 columns. I was able to convert the CSV to a JSON using one of the myriad of online converters, but the result is what I believed is termed a 'flat JSON file'.
Here is the CSV header row:
F1,F2,F3.1.F1,F3.1.F2,F3.1.F3,F3.1.F4,...F3.10.F1,F3.10.F2,F3.10.F3,F3.10.F4,F4
Here is my JSON schema:
{
"$schema": "http://json-schema.org/schema#",
"$id": "./.schema.json",
"title": "",
"description": "",
"type": "object",
"properties": {
"F1": {
"description": "",
"type": "string"
},
"F2": {
"description": "",
"type": "string"
},
"F3": {
"description": "",
"type": "array",
"items": {
"description": "",
"type": "object",
"properties": {
"F3.F1": {
"description": "",
"type": "string"
},
"F3.F2": {
"description": "",
"type": "string"
},
"F3.F3": {
"description": "",
"type": "string"
},
"F3.F4": {
"description": "",
"type": "string"
}
},
"required": [
"F3.F1",
"F3.F2",
"F3.F3",
"F3.F4"
]
},
"numItems": 10,
"unique": false
},
"F4": {
"description": "",
"type": "string"
}
},
"required": [
"F1",
"F2",
"F3",
"F4"
],
"additionalProperties": false
}
From the CSV->JSON conversion, my JSON file looks like:
[
{
"F1": 2429546524130460000,
"F2": 2429519276857919500,
"F3.1.F1": 2428316170619109000,
"F3.1.F2": 0.0690932185744956,
"F3.1.F3": 2.6355498567408557,
"F3.1.F4": 0.4369495787854096,
...
"F3.10.F1": 2429415922764859400,
"F3.10.F2": 0.15328371980044203,
"F3.10.F3": 2.677944208300451,
"F3.10.F4": 0.31036472544281585,
"F4": 0.16889514829995647
},
... //repeated 2,509 times
{
"F1": 1143081876266241000,
"F2": 1143588785487818100,
"F3.1.F1": 1141377392726037800,
"F3.1.F2": 1.332366799133926,
"F3.1.F3": 0.24878185970548322,
"F3.1.F4": 1.560443994684636,
...
"F3.10.F1": "XXX",
"F3.10.F2": "XXX",
"F3.10.F3": "XXX",
"F3.10.F4": "XXX",
"F4": 2.2916768389567497
}
]
Clearly, making the necessary changes 2,511 times is impractical, so I am hoping there is a method to make the changes automatically. I can code, but I could not find any specific solutions anywhere to go from a CSV to a JSON with the JSON output matching a specific JSON schema. Preferably, I would like a solution that is not restricted to just converting this one set of data to this one specific format, i.e., a general solution that could be used with a different CSV and different JSON schema.

Reference Another Schema in Web Validator

I'm trying to learn JSON schema and choose the web validator to validate some examples. It works fine except when I try to reference another schema. The schemas are saved in my machine and I don't know how to make it works, or even if this kind of operation works in web validator.
When I try to use the validator with reference, I get the following error:
Error parsing schema
Message: Error when resolving schema reference 'file:///D:/schema/address.schema.json'. Path 'properties.shipping_address', line 12, position 25.
[![error image][1]][1]
[1]: https://i.stack.imgur.com/6z2pY.png
Any help will be appreciated!
The schemas and JSON used are show below:
{
"$id": "D:/schema/customer.schema.json",
"$schema": "https://json-schema.org/draft/2019-09/schema",
"type": "object",
"properties": {
"first_name": {
"type": "string"
},
"last_name": {
"type": "string"
},
"shipping_address": {
"$ref": "D:/schema/address.schema.json"
},
"billing_address": {
"$ref": "D:/schema/address.schema.json"
}
},
"required": [
"first_name",
"last_name",
"shipping_address",
"billing_address"
]
}
---
{
"$id": "D:/schema/address.schema.json",
"$schema": "https://json-schema.org/draft/2019-09/schema",
"type": "object",
"properties": {
"street_address": {
"type": "string"
},
"city": {
"type": "string"
},
"state": {
"type": "string"
}
},
"required": [
"street_address",
"city",
"state"
]
}
--- JSON
{
"first_name": "first",
"last_name": "last",
"shipping_address": {
"street_address": "street",
"city": "city",
"state": "ST"
},
"billing_address": {
"street_address": "street",
"city": "city",
"state": "ST"
}
}

Json parse is giving null values

I have a logic app which gets Address info using Bind Rest Api. I received a nested json object. When I tried to parse it, I'm getting null values in dynamix syntax box.
Below is the json object that I get, however when I use these as properties after parsing, they are marked as null:
{
"address": {
"addressLine": "1-11-252, Begumpet Road",
"adminDistrict": "TS",
"adminDistrict2": "Hyderabad",
"countryRegion": "India",
"formattedAddress": "1-11-252, Begumpet Road, Hyderabad, TS 500016",
"intersection": {
"baseStreet": "Begumpet Road",
"secondaryStreet1": "Chikoti Garden No-4 Road",
"intersectionType": "Near",
"displayName": "Begumpet Road and Chikoti Garden No-4 Road"
},
"locality": "Hyderabad",
"neighborhood": "Begumpet",
"postalCode": "500016",
"countryRegionIso2": "IN"
}
}
Based on your information, here's what I did:
Created a Logic App in the Azure portal
Selected the "When a HTTP request is received" trigger
Clicked "Use sample payload to generate schema" link and pasted your example data (SEE BELOW!)
Added a "Response" action
Defined the response's body to be:
{
"address": baseStreet
}
Where baseStreet is a reference to the baseStreet dynamic content.
The result:
IMPORTANT!
Please be advised that the Request Body JSON Schema is different than the payload:
{
"type": "object",
"properties": {
"address": {
"type": "object",
"properties": {
"addressLine": {
"type": "string"
},
"adminDistrict": {
"type": "string"
},
"adminDistrict2": {
"type": "string"
},
"countryRegion": {
"type": "string"
},
"formattedAddress": {
"type": "string"
},
"intersection": {
"type": "object",
"properties": {
"baseStreet": {
"type": "string"
},
"secondaryStreet1": {
"type": "string"
},
"intersectionType": {
"type": "string"
},
"displayName": {
"type": "string"
}
}
},
"locality": {
"type": "string"
},
"neighborhood": {
"type": "string"
},
"postalCode": {
"type": "string"
},
"countryRegionIso2": {
"type": "string"
}
}
}
}
}
When running with the Parse JSON step:

ajv does not return when validating a valid json

I have the following schema:
{
"$schema": "http://json-schema.org/schema#",
"$id": "http://api.hobnob.social/schemas/users/create.json",
"title": "Create User Schema",
"description": "For validating client-provided create user object",
"type": "object",
"properties": {
"email": {
"type": "string",
"format": "email"
},
"password": { "type": "string" },
"profile": { "$ref": "profile.json#" }
},
"required": ["email", "password"],
"additionalProperties": false
}
{
"$schema": "http://json-schema.org/schema#",
"$id": "http://api.hobnob.social/schemas/users/profile.json",
"title": "User Profile Schema",
"description": "For validating client-provided user profile object when creating and/or updating an user",
"type": "object",
"properties": {
"bio": { "type": "string" },
"summary": { "type": "string" },
"name": {
"type": "object",
"properties": {
"first": { "type": "string" },
"last": { "type": "string" },
"middle": { "type": "string" }
},
"additionalProperties": false
}
},
"additionalProperties": false
}
I am using ajv to validate against it. I am getting the expected results in almost all cases. But when validating a json with either the bio or summary fields included (with type of string), no response comes from ajv at all.
E.g. I attempt to validate
{
"email": "e#ma.il",
"password": "password",
"profile": {
"name": {
"first": "firstname"
},
"bio":"this is a bio"
}
}
and no response at all comes back.
I tried consolidating the schema but that made no difference. I'm hoping I have made some simple beginner mistake that someone may spot! I have spent many hours trying to work out what is going wrong, but after all my debugging I am no further forward.
I got this working somehow, but not sure why it started working.
In my test script I added a line to delete the test index from elasticsearch. After that, all tests passed. I then removed the new line from my test script to see if it would stop working again, but it didn't.
I'm guessing the problem was somehow related to elasticsearch...