Can we cross-reference entities in different service paths using Relationships? - fiware

We are using FIWARE Orion NGSI V2 version. We are trying to create a data model where entities exist in different service paths and these entities are connected to each other through relationships.
We checked FIWARE documentation and did not find any answers on how to achieve this.
Please find attached the pictorial representation of what we are trying to achieve. Also the version of FIWARE Orion we are using. "Connected To" relationship to be created between entities in different service paths. Can you please guide us on how to achieve this.
{
"orion" : {
"version" : "2.4.0-next",
"uptime" : "0 d, 0 h, 1 m, 7 s",
"git_hash" : "4f26834ca928e468b091729d93dabd22108a2690",
"compile_time" : "Tue Mar 31 16:21:23 UTC 2020",
"compiled_by" : "root",
"compiled_in" : "3369cff2fa4c",
"release_date" : "Tue Mar 31 16:21:23 UTC 2020",
"doc" : "https://fiware-orion.rtfd.io/"
}
}

You only need a trio of attributes like this one:
XXXSource, for de ID of the entity
XXXSourceType, for the type of the entity
XXXSourcePath, for the service path of the entity
Where XXX is a token specifying the name of the relationship if you need to distinguish many of them. For instance:
managerSource
managerSourceType
managerSourcePath
Other naming variants are possible, e.g.
refXXX
refXXXType
refXXXPath

The fiware-servicepath header is meant to logically divide a system in a hierarchical manner, By default the / hierarchy is assumed.
If you create the following:
curl -L -X POST 'http://localhost:1026/v2/entities/' \
-H 'fiware-servicepath: /pigs' \
-d {
"id": "urn:ngsi-ld:Animal:001",
"type": "Animal",
}'
curl -L -X POST 'http://localhost:1026/v2/entities/' \
-H 'fiware-servicepath: /cows' \
-d {
"id": "urn:ngsi-ld:Animal:002",
"type": "Animal",
}'
You can retrieve cows only by requesting the following:
curl -L -X GET 'http://localhost:1026/v2/entities' \
-H 'fiware-servicepath: /cows'
You can retrieve cows and pigs by requesting:
curl -L -X GET 'http://localhost:1026/v2/entities' \
-H 'fiware-servicepath: /#'
Therefore if your Relationship is cross hierarchy, you just need to supply a fiware-servicepath with a wildcard service-path /# when retrieving entities.
Note that if the data is found under a longer service-path (e.g. /animals/cows and animals/pigs) then you could use /animals/# or /# as necessary.

I would not recommend at all using the FIWARE Service Path. You can very easily end up with a system with duplicate ids and a total mess.
In a linked data and distributed scenario you can have pointers to entities that live within multiple brokers or systems and just use URIs to point to them through Relationships.

Related

Bash & cURL: Retrieve JSON from web API and search it for specific key:value pairs

I'm cURLing a web API for an application/json response, that response is a set of key value pairs, like this:
{"id":89,"name":"User saved 2018-07-03 12:01:47.337483","create_time":1530644507337,"auto":false,"recovered":false}
{"id":49,"name":"User saved 2018-05-24 12:33:53.927798","create_time":1527190433927,"auto":false,"recovered":false}
{"id":199,"name":"Daily backup 2018-10-22 02:37:37.332271","create_time":1540201057332,"auto":true,"recovered":false}
etc, etc...
I'd like to iterate through this response and find the highest value integer for the "id" key then save that as a variable. If the above was my whole JSON I'd want to end up with variable=199.
Doing something like this:
MY_VARIABLE=$(curl -k -X GET --header "Accept: application/json" \
--header "Authorization: MyAPITarget apikey=${MY_APIKEY}" \
"https://targetserver/api/methodImCalling" |
The output of that is the JSON above. Can I pipe that output into an array and iterate through it but only look for the value of "id" then do something similar to a:
for (i = 0; i < id.length; i++)
I've only been working with code a short while and most my background at this point is JS, trying to make the connection here for bash. I'm trying to avoid using any "installed" tools whatsoever which is why I'm using bash, I'd like this script to run "out of the box" on any linux / unix platform. Any tips? Thanks!
It's probably a separate installation, but the tool you want is jq:
max_id=$(curl ... | jq -s 'map(.id) | max')
The standard tools that one can expect to be pre-installed simply aren't suitable for working with JSON.
While not standard, any machine that has curl installed is likely to have Python installed, and you can use its standard json module to process the JSON properly. Here's a somewhat ungainly one-liner:
curl ... |
python -c 'import json,sys; x="[%s]"%(",".join(sys.stdin),); print(max(y["id"] for y in json.loads(x)))'
Other non-standard but common languages (Perl, Ruby, etc) probably also have built-in ways to consume JSON.

How to use the dbicdump to only dump specific table

I just need to dump specific tables from my database such that these specific tables (3 tables to be exact out of 200 tables) will now be implemented by DBIx::Class::Schema.
Here is the command from the docs (https://metacpan.org/pod/dbicdump):
dbicdump -o dump_directory=./lib -o components='["InflateColumn::DateTime"]' -o preserve_case=1 MyApp::Schema dbi:mysql:database=database_name user pass;
I tried appending the table name after the database_name but no luck, it still dumps all the tables in the specified database. Need help. I can't find anything in the docs.
Also Out of topic question :
What does these means? -o components='["InflateColumn::DateTime"]' -o preserve_case=1 I also cant find their explanation in the docs.
Thanks
You can pass the option constraint to the underlying DBIx::Class::Schema::Loader instance so it only selects certain tables. The documentation is a bit vague on this.
These can be specified either as a regex (preferrably on the qr//
form), or as an arrayref of arrayrefs. Regexes are matched against the
(unqualified) table name, while arrayrefs are matched according to
"moniker_parts".
For example:
db_schema => [qw(some_schema other_schema)],
moniker_parts => [qw(schema name)],
constraint => [
[ qr/\Asome_schema\z/ => qr/\A(?:foo|bar)\z/ ],
[ qr/\Aother_schema\z/ => qr/\Abaz\z/ ],
],
In this case only the tables foo and bar in some_schema and baz in
other_schema will be dumped.
So what you need to pass to dbicdump would look something like this.
dbicdump \
-o dump_directory=./lib \
-o components='["InflateColumn::DateTime"]' \
-o preserve_case=1 \
-o constraint='qr/^(?:foo|bar|baz)$/' \
MyApp::Schema dbi:mysql:database=database_name user pass;
That will only give you the tables foo, bar and baz. You need the quoted regular expression without array refs if there is only one schema, and you don't want to use manually preset monikers (which are names used for tables in the generated Schema class).

Passing key value from json in CURL request (command line)

I am using a REST API to move a user into a group, but need to append the group name to the URL when I run the command.
So the path to my REST API is:
http://server:8080/rest/api/2/group/user?groupname
And it's expecting "groupname" to be passed as "groupname="Name%20Of%20Group"
i.e.
http://server:8080/rest/api/2/group/user?groupname="Name%20Of%20Group"
The full command I'm running on Windows is
curl -u name:pass -X POST --data #add_user.txt -H "Content-Type: application/json" http://server:8080/rest/api/2/group/user?groupname
add_user.txt is structured like this
{
"name": "tim",
"groupname": "MY%20TEST%20GROUP"
},
{
"name": "carol",
"groupname": "MY%20TEST%20GROUP"
}
It's looping through the names I believe, but I want CURL to pick up on each "groupname" defined in the file.
Any ideas on how I can do this?
each curl call makes a single request.
looks like you need to change from POST to GET and use the -G, --get option to add the values as parameters after the ?
See How do I use cURL to perform multiple simultaneous requests?
HTH

Range query in ElasticSearch (GET without body)

So very basic question about elasticsearch which the docs not answer very clearly (because they seem to go into many advanced details but miss the basic ones!).
Example: range query
http://www.elasticsearch.org/guide/reference/query-dsl/range-query.html
Doesn't tell how to PERFORM the range, is it via the search endpoint?
And if it is, then how to do it via querystring? I mean, I want to do a GET, not a POST (because it's a query, not an insertion/modification). However the documention for GET requests doesn't tell how to use JSON like in the Range sample:
http://www.elasticsearch.org/guide/reference/api/search/uri-request.html
What am I missing?
Thanks
Use the Lucene query syntax:
curl -X GET 'http://localhost:9200/my_index/_search?q=my_field:[0+TO+25]&pretty'
Let's assume we have an index
curl -XPUT localhost:9200/test
And some documents
curl -XPUT localhost:9200/test/range/1 -d '{"age": 9}'
curl -XPUT localhost:9200/test/range/2 -d '{"age": 12}'
curl -XPUT localhost:9200/test/range/3 -d '{"age": 16}'
Now we can query these documents within a certain range via
curl -XGET 'http://localhost:9200/test/range/_search?pretty=true' -d '
{
"query" : {
"range" : {
"age" : {
"from" : "10",
"to" : "20",
"include_lower" : true,
"include_upper": true
}
}
}
}
'
This will return the documents 2 and 3.
I'm not sure if there is a way to perform these kind of complex queries via URI request, though.
Edit: Thanks to karmi here is the solution without JSON request:
curl -XGET --globoff 'localhost:9200/test/range/_search?q=age:["10"+TO+"20"]&pretty=true'
Replying to myself thanks to #javanna:
In the RequestBody section of the Search docs:
http://www.elasticsearch.org/guide/reference/api/search/request-body.html
At the end, it says:
The rest of the search request should be passed within the body itself. The body content can also be passed as a REST parameter named source.
So I guess that I need to use the search endpoint with the source attribute to pass json.

ElasticSearch, specific field not returning

I'm stuck with a little elasticsearch problem. I'm new to elasticsearch and don't know why this doesn't work.
curl -XPOST 'http://myhost.nl:9200/my_index/test/_search?pretty=true' -d '{ "fields": ["message"] }'
I don't get any field back. The field "message" does exist and realy looks like the example on the elasticsearch site. http://www.elasticsearch.org/guide/reference/api/search/fields.html
Can anybody see what I'm missing?
Your query would have worked if this field was stored. But since it's not stored and only available as part of source, you need to specify full path to it. Try:
curl -XPOST 'http://myhost.nl:9200/my_index/test/_search?pretty=true' -d '{ "fields": ["tweet.message"] }'