could i write like this?
function (doc,meta) {
emit([[doc.range1, doc.range2],[doc.range3, doc.range4]], doc.comment);
}
i just want to find data use two range.I found a example that is
emit([10.9, 48.4, [1000, 2000]], null);
the range [1000,2000] does it can filter data?
The following link will help: http://developer.couchbase.com/documentation/server/4.0/sdks/dotnet-2.2/geo-spatial-views.html
And remember, spatial feature is experimental before version 4.0.
For use with python, in my case, it provides SpatialQuery object.
(from couchbase.views.params import SpatialQuery)
supposing you have a json document shaped this way
{
"resourceId": "3c0d9906-4e88-4b12-b96b-9d7377a503a9",
"_type": "com.sample.model.Position",
"updated": 1510790620782,
"point": {
"coordinates": [15.704621,40.2755],
"type": "Point"
}
}
you can write a spatial view this way
function (doc) {
if (typeof doc.point !== 'undefined' && doc._type == "com.sample.model.Position") {
var date = Date.parse(doc.updated) / 1000;
emit(doc.point, doc.resourceId);
}
}
you can use the browser to design your docuement:
go to http://localhost:8091
enter index -> view menu
from there you can test the view with real document from your data
this create the index; to query the index you can use the geo funcions even with rest as described by
official documentation
hope it helps
Related
Trying to figuring out how to deserialize this kind of json in talend components :
{
"ryan#toofr.com": {
"confidence":119,"email":"ryan#toofr.com","default":20
},
"rbuckley#toofr.com": {
"confidence":20,"email":"rbuckley#toofr.com","default":15
},
"ryan.buckley#toofr.com": {
"confidence":18,"email":"ryan.buckley#toofr.com","default":16
},
"ryanbuckley#toofr.com": {
"confidence":17,"email":"ryanbuckley#toofr.com","default":17
},
"ryan_buckley#toofr.com": {
"confidence":16,"email":"ryan_buckley#toofr.com","default":18
},
"ryan-buckley#toofr.com": {
"confidence":15,"email":"ryan-buckley#toofr.com","default":19
},
"ryanb#toofr.com": {
"confidence":14,"email":"ryanb#toofr.com","default":14
},
"buckley#toofr.com": {
"confidence":13,"email":"buckley#toofr.com","default":13
}
}
This JSON comes from the Toofr API where documentation can be found here .
Here the actual sitation :
For each line retreived in the database, I call the API and I got this (the first name, the last name and the company change everytime.
Does anyone know how to modify the tExtractJSONField (or use smthing else) to show the results in tLogRow (for each line in the database) ?
Thank you in advance !
EDIT 1:
Here's my tExtractJSONfields :
When using tExtractJSONFields with XPath, you need
1) a valid XPath loop point
2) valid XPath mapping to your structure relative to the loop path
Also, when using XPath with Talend, every value needs a key. The key cannot change if you want to loop over it. Meaning this is invalid:
{
"ryan#toofr.com": {
"confidence":119,"email":"ryan#toofr.com","default":20
},
"rbuckley#toofr.com": {
"confidence":20,"email":"rbuckley#toofr.com","default":15
},
but this structure would be valid:
{
"contact": {
"confidence":119,"email":"ryan#toofr.com","default":20
},
"contact": {
"confidence":20,"email":"rbuckley#toofr.com","default":15
},
So with the correct data the loop point might be /contact.
Then the mapping for Confidence would be confidence (the name from the JSON), the mapping for Email would be email and vice versa for default.
EDIT
JSONPath has a few disadvantages, one of them being you cannot go higher up in the hierarchy. You can try finding out the correct query with jsonpath.com
The loop expression could be $.*. I am not sure if that will satisfy your need, though - it has been a while since I've been using JSONPath in Talend because of the downsides.
I have been ingesting some complex json structures and did this via minimal json libraries, and tjava components within talend.
Neo4j Version: 3.2.2
Operating System: Ubuntu 16.04
I use getDegree() function in mapping.json file, but the return would always be null; I'm using the dataset neo4j tutorial Movie/Actor dataset.
Output from elasticsearch request
mapping.json
{
"defaults": {
"key_property": "uuid",
"nodes_index": "default-index-node",
"relationships_index": "default-index-relationship",
"include_remaining_properties": true
},
"node_mappings": [
{
"condition": "hasLabel('Person')",
"type": "getLabels()",
"properties": {
"getDegree": "getDegree()",
"getDegree(type)": "getDegree('ACTED_IN')",
"getDegree(direction)": "getGegree('OUTGOING')",
"getDegree('type', 'direction')": "getDegree('ACTED_IN', 'OUTGOING')",
"getDegree-degree": "degree"
}
}
],
"relationship_mappings": [
{
"condition": "allRelationships()",
"type": "type",
}
]
}
Also if I use isOutgoing(), isIncoming(), otherNode function in relationship_mappings properties part, elasticsearch would never load the relationship data from neo4j. I think I probably have some misunderstanding of this sentence only when one of the participating nodes "looking" at the relationship is provided on this page https://github.com/graphaware/neo4j-framework/tree/master/common#inclusion-policies
mapping.json
{
"defaults": {
"key_property": "uuid",
"nodes_index": "default-index-node",
"relationships_index": "default-index-relationship",
"include_remaining_properties": true
},
"node_mappings": [
{
"condition": "allNodes()",
"type": "getLabels()"
}
],
"relationship_mappings": [
{
"condition": "allRelationships()",
"type": "type",
"properties": {
"isOutgoing": "isOutgoing()",
"isIncomming": "isIncomming()",
"otherNode": "otherNode"
}
}
]
}
BTW, is there any page that list all of the functions that we can use in mapping.json? I know two of them
github.com/graphaware/neo4j-framework/tree/master/common#inclusion-policies
github.com/graphaware/neo4j-to-elasticsearch/blob/master/docs/json-mapper.md
but it seems there are more, since I can use getType(), which hasn't been listed in any of the above pages.
Please let me know if I can provide any further help to solve the problem
Thanks!
The getDegree() function is not available to use, in contrary to getType(). I will explain why :
When the mapper (the part responsible to create a node or relationship representation as ES document ) is doing its job, it receive a DetachedGraphObject being a detached node or relationship.
The meaning of detached is that it is happening outside of a transaction and thus query operations are not available against the database anymore. The getType() is available because it is part of the relationship metadata and it is cheap, however if we would want to do the same for getDegree() this can be seriously more costly during the DetachedObject creation (which happen in a tx) depending on the number of different types etc.
This is however something we are working on, by externalising the mapper in a standalone java application coupled with a broker like kafa, rabbit,.. between neo and this app. We would not, however offer the possibilty to requery the graph in the current version of the module as it can have serious performance impacts if the user is not very careful.
As last, the only suggestion I can give you is to keep a property on your node with the updates of degrees you need to replicate to ES.
UPDATE
Regarding this part of the documentation :
For Relationships only when one of the participating nodes "looking" at the relationship is provided:
This is used only when not using the json definition, so you can use one or the other. the json definition has been added later as addition and both cannot be used together.
For answering this part, it means that the nodes of the incoming or outgoing side, depending on the definition, should be included in the inclusion policy for nodes, like hasLabel('Employee') || hasProperty('form') || getProperty('age', 0) > 20 . If you have an allNodes policy then it is fine.
what is the best approach to capture from the following array?
i only need to capture the value of ANY 'beginDate', e.g: 2017-05-01T08:30:00 could be a valid one in below example
i need to make sure the 'beschikbaar' = TRUE for the date that i'm capturing
i tried using json path extractor with similar lines: $..[?(#.beschikbaar == 'true')].beginDate but i'm facing syntax errors that i cant fix due to my limited regex/json path knowledge
the example array is;
{
"data":
[
[
{
"beginDate":"2017-05-01T08:00:00",
"eindDate":null,
"beschikbaar":false
},
{
"beginDate":"2017-05-01T08:15:00",
"eindDate":null,
"beschikbaar":false
},
{
"beginDate":"2017-05-01T08:30:00",
"eindDate":"2017-05-01T10:30:00+02:00",
"beschikbaar":true
},
{
"beginDate":"2017-05-01T08:45:00",
"eindDate":"2017-05-01T10:45:00+02:00",
"beschikbaar":true
},
{
"beginDate":"2017-05-01T09:00:00",
"eindDate":"2017-05-01T11:00:00+02:00",
"beschikbaar":true
},
{
"beginDate":"2017-05-01T09:15:00",
"eindDate":"2017-05-01T11:15:00+02:00",
"beschikbaar":true
},
{
"beginDate":"2017-05-01T09:30:00",
"eindDate":"2017-05-01T11:30:00+02:00",
"beschikbaar":true
},
{
"beginDate":"2017-05-01T09:45:00",
"eindDate":"2017-05-01T11:45:00+02:00",
"beschikbaar":true
},
{
"beginDate":"2017-05-01T10:00:00",
"eindDate":"2017-05-01T12:00:00+02:00",
"beschikbaar":true
},
Don't use regular expressions for JSON data, JMeter provides JSON Extractor designed to work with JSON data via JSON Path Language so you should be able to get your "beginDate" with the query like:
$..[?(#.beschikbaar == true)].beginDate
Demo:
Check out JMeter's JSON Path Extractor Plugin - Advanced Usage Scenarios article for more detailed explanation and few more examples.
You can try this
(?s)\{.*?\"beginDate\":\"([^{]*?)\"[^{]+\"beschikbaar\":true.*?\}
(?s) is single-line modifier which makes . match the line break
You can test it at http://www.regexplanet.com/advanced/java/index.html
And set Template to $1$ means using the first group
I am trying to create a view to group on a particular attribute inside an array. However, the below map function is not returning any result.
JSON Document Structure :
{
"jsontype": "survey_response",
"jsoninstance": "xyz",
"jsonlanguage": "en_us",
"jsonuser": "test#test.com",
"jsoncontact": "test#mayoclinic.com",
"pages": [
{
q-placeholder": "q1-p1",
q:localized": "q1-localized-p1",
q-answer-placeholder": "jawaabu121",
q-answer-localized": "localized jawaabu1"
},
{
q-placeholder": "q2-p2",
q:localized": "q2-localized-p2",
q-answer-placeholder": "jawaabu221",
q-answer-localized": "localized jawaabu2"
},
{
"q-placeholder": "q3-p3",
"q:localized": "q3-localized-p3",
"q-answer-placeholder": "jawaabu313",
"q-answer-localized": "localized jawaabu3"
}
]
}
Map Function :
function(doc, meta){
emit(doc.jsoninstance,[doc.pages[0].q-placeholder, doc.pages[0].q-localized,doc.pages[0].q-answer-placeholder,q-answer-localized]);
}
It looks like you made a typo at the end of your emit statement:
doc.pages[0].q-answer-placeholder,q-answer-localized.
Instead q-answer-localized should be changed to doc.pages[0].q-answer-localized.
Further to this it seems that you have defined a field as q-localized in your emit statement, but actually according to the sample document that you posted this should actually be q:localized, I assume that this was a mistake in the snippet of the document and not the emit statement, but if not then will also need amending.
I would imagine errors like this would be flagged up in the view engine's map-reduce error log, in future you should check this log so that you will be able to debug errors like this yourself.
The location of the mapreduce_errors log can be found in the Couchbase documentation
I would like to generate JAVA classes from a given JSON Schema draft 4 version
I evaluated couple of tools and jsonschema2pojo was found to be useful. But it supports json schema draft-3 version only(although json schema draft 4 is in their roadmap).
Can anyone suggest me a tool or a way to generate java classes from a json schema (compliant to json schema draft4)
?
Thanks in advance.
You might try cog, a general purpose code generator written in Ruby. I put a simple project on github called json2java which demonstrates how cog might be used to generate Java classes from json data.
Not sure exactly what you want to do, but here is what I assumed. The json data would look something like this
{
"classname": "Sample",
"methods": [
{
"name": "foo",
"rtype": "void",
"params": [
{
"name": "arg1",
"type": "int"
}
]
},
{
"name": "bar",
"rtype": "int",
"params": []
}
]
}
And the corresponding Java class would look something like this
public class Sample {
void foo(int arg1) {
// keep: foo {
// While the interface in this example is generated,
// the method bodies are preserved between multiple invocations
// of the generator.
// It doesn't have to be done this way, the method bodies can be
// generated aswell, all depends on what your json data encodes
// keep: }
}
int bar() {
// keep: bar {
return 1;
// keep: }
}
}
If you want to try cog, install it like this gem install cog, and run generators like this cog gen. Check out the cog homepage for documentation.