I wonder if there is a possibility to express the following code in a more readable manner. Maybe without SHACL-SPARQL. Is it possible to use the function sh:filterShape in this context?
#prefix sh: <http://www.w3.org/ns/shacl#> .
#prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
#prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
#prefix : <http://www.semanticweb.org#> .
#prefix owl: <http://www.w3.org/2002/07/owl#> .
:RoomNumberShape a sh:NodeShape ;
sh:target [
a sh:SPARQLTarget ;
sh:prefixes [
sh:declare [
sh:prefix "" ;
sh:namespace "http://www.semanticweb.org#"^^xsd:anyURI ;
] ;
sh:declare [
sh:prefix "rdf" ;
sh:namespace "http://www.w3.org/1999/02/22-rdf-syntax-ns#"^^xsd:anyURI ;
]
] ;
sh:select """
SELECT ?node
WHERE {
?node :has_RD :RD200 ;
rdf:type :Room ;
}
""" ;
] ;
sh:property [
sh:path :has_RoomNumber ;
sh:minCount 1 ;
sh:maxCount 1 ;
] .```
Related
I'm trying to map this JSON file to RDF, but I probably can not iterate correctly to get the values of "value", which are inside the measures array.
JSON:
{
"status": 0,
"body": {
"updatetime": 1528904042,
"timezone": "Europe\/Rome",
"measuregrps": [{
"grpid": 1154218424,
"attrib": 2,
"date": 1528902698,
"category": 1,
"brand": 1,
"modified": 1528902700,
"deviceid": null,
"measures": [{
"value": 7000,
"type": 11,
"unit": -2,
"algo": 0,
"fw": 0,
"fm": 131
}]
},
{
"grpid": 1154218987,
"attrib": 2,
"date": 1528902745,
"category": 1,
"brand": 1,
"modified": 1528902747,
"deviceid": null,
"measures": [{
"value": 7200,
"type": 11,
"unit": -2,
"algo": 0,
"fw": 0,
"fm": 131
}]
}
]
}
}
RML:
#prefix rr: <http://www.w3.org/ns/r2rml#>.
#prefix rml: <http://semweb.mmlab.be/ns/rml#> .
#prefix ql: <http://semweb.mmlab.be/ns/ql#> .
#prefix xsd: <http://www.w3.org/2001/XMLSchema#>.
#prefix fo: <http://purl.org/ifo/#>.
###### Fitbit MAPPING #######
<#FitbitRestingHeartRate>
rml:logicalSource [
rml:source "provaJson.json";
rml:referenceFormulation ql:JSONPath;
rml:iterator "$.body.measuregrps";
];
rr:subjectMap [
rr:template "http://ifo.com/{grpid}";
rr:class fo:HeartRate;
];
rr:predicateObjectMap [
rr:predicate fo:hasTemporalRelationshipToPhysicalActivity;
rr:objectMap [
rr:constant fo:AtRest;
];
];
rr:predicateObjectMap [
rr:predicate fo:hasMeasure;
rr:objectMap [
rr:parentTriplesMap <#MeasureHeartRate>;
];
].
<#MeasureHeartRate>
rml:logicalSource [
rml:source "provaJson.json";
rml:referenceFormulation ql:JSONPath;
rml:iterator "$.body.measuregrps";
];
rr:subjectMap [
rr:template "http://ifo.com/{grpid}";
rr:class fo:Measure;
rml:iterator "$.body.measuregrps";
];
rr:predicateObjectMap [
rr:predicate fo:hasNumericalValue;
rr:objectMap [
rml:reference "#.measures.value";
rr:datatype xsd:float;
];
];
rr:predicateObjectMap [
rr:predicate fo:hasDescriptiveStatistic;
rr:objectMap [
rr:constant fo:average;
];
];
rr:predicateObjectMap [
rr:predicate fo:hasUnit;
rr:objectMap [
rr:constant fo:bpm;
];
].
Thanks for your help,
Chiara
Adapting the JSON Path expression allows RML processors to retrieve the heart
rate values.
For both TriplesMaps I changed them to $.body.measuregrps.[*] which iterate
over each measuregrps entry.
I added a rr:TriplesMap for each TriplesMap to make sure that an RML
processor knows that the RDF describes a TriplesMap:
<#MeasureHeartRate>
a rr:TriplesMap;
All heart rate values where matched to every measurement.
I added a rr:joinCondition which only joins the heart rate value with the
right measurement when grpid values are equal:
rr:predicateObjectMap [
rr:predicate fo:hasMeasure;
rr:objectMap [
rr:parentTriplesMap <#MeasureHeartRate>;
rr:joinCondition [
rr:child "grpid";
rr:parent "grpid";
];
];
].
I changed the IRI of the heart rate measurement mapping since the IRI was
not unique which resulted into a loop:
<http://ifo.com/1154218987> <http://purl.org/ifo/#hasMeasure> <http://ifo.com/1154218987>
The new IRI format also supports other types of measurements besides the heart
rate.
Mapping rules:
#base <http://example.org> .
#prefix rr: <http://www.w3.org/ns/r2rml#> .
#prefix rml: <http://semweb.mmlab.be/ns/rml#> .
#prefix ql: <http://semweb.mmlab.be/ns/ql#> .
#prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
#prefix fo: <http://purl.org/ifo/#> .
<#FitbitRestingHeartRate>
a rr:TriplesMap;
rml:logicalSource [
rml:source "provaJson.json";
rml:referenceFormulation ql:JSONPath;
rml:iterator "$.body.measuregrps.[*]";
];
rr:subjectMap [
rr:template "http://ifo.com/{grpid}";
rr:class fo:HeartRate;
];
rr:predicateObjectMap [
rr:predicate fo:hasTemporalRelationshipToPhysicalActivity;
rr:objectMap [
rr:constant fo:AtRest;
];
];
rr:predicateObjectMap [
rr:predicate fo:hasMeasure;
rr:objectMap [
rr:parentTriplesMap <#MeasureHeartRate>;
rr:joinCondition [
rr:child "grpid";
rr:parent "grpid";
];
];
].
<#MeasureHeartRate>
a rr:TriplesMap;
rml:logicalSource [
rml:source "provaJson.json";
rml:referenceFormulation ql:JSONPath;
rml:iterator "$.body.measuregrps.[*]";
];
rr:subjectMap [
rr:template "http://ifo.com/{grpid}/{date}/{measures.[*].type}";
rr:class fo:Measure;
];
rr:predicateObjectMap [
rr:predicate fo:hasNumericalValue;
rr:objectMap [
rml:reference "measures.[*].value";
rr:datatype xsd:float;
];
];
rr:predicateObjectMap [
rr:predicate fo:hasDescriptiveStatistic;
rr:objectMap [
rr:constant fo:average;
];
];
rr:predicateObjectMap [
rr:predicate fo:hasUnit;
rr:objectMap [
rr:constant fo:bpm;
];
].
Output:
<http://ifo.com/1154218424> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://purl.org/ifo/#HeartRate>.
<http://ifo.com/1154218424> <http://purl.org/ifo/#hasTemporalRelationshipToPhysicalActivity> <http://purl.org/ifo/#AtRest>.
<http://ifo.com/1154218424> <http://purl.org/ifo/#hasMeasure> <http://ifo.com/1154218424/1528902698/11>.
<http://ifo.com/1154218987> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://purl.org/ifo/#HeartRate>.
<http://ifo.com/1154218987> <http://purl.org/ifo/#hasTemporalRelationshipToPhysicalActivity> <http://purl.org/ifo/#AtRest>.
<http://ifo.com/1154218987> <http://purl.org/ifo/#hasMeasure> <http://ifo.com/1154218987/1528902745/11>.
<http://ifo.com/1154218424/1528902698/11> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://purl.org/ifo/#Measure>.
<http://ifo.com/1154218424/1528902698/11> <http://purl.org/ifo/#hasNumericalValue> "7000"^^<http://www.w3.org/2001/XMLSchema#float>.
<http://ifo.com/1154218424/1528902698/11> <http://purl.org/ifo/#hasDescriptiveStatistic> <http://purl.org/ifo/#average>.
<http://ifo.com/1154218424/1528902698/11> <http://purl.org/ifo/#hasUnit> <http://purl.org/ifo/#bpm>.
<http://ifo.com/1154218987/1528902745/11> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://purl.org/ifo/#Measure>.
<http://ifo.com/1154218987/1528902745/11> <http://purl.org/ifo/#hasNumericalValue> "7200"^^<http://www.w3.org/2001/XMLSchema#float>.
<http://ifo.com/1154218987/1528902745/11> <http://purl.org/ifo/#hasDescriptiveStatistic> <http://purl.org/ifo/#average>.
<http://ifo.com/1154218987/1528902745/11> <http://purl.org/ifo/#hasUnit> <http://purl.org/ifo/#bpm>.
Note: I contribute to RML and its technologies.
Given the input in this form
[
{
"DIR" : "/foo/bar/a/b/c",
"OUT" : "/foo/bar/x/y/z",
"ARG" : [ "aaa", "bbb", "/foo/bar/a", "BASE=/foo/bar" ]
},
{
"DIR" : "/foo/baz/d/e/f",
"OUT" : "/foo/baz/x/y/z",
"ARG" : [ "ccc", "ddd", "/foo/baz/b", "BASE=/foo/baz" ]
},
{
"foo" : "bar"
}
]
I'm trying to find out how to make jq transform that into this:
[
{
"DIR" : "BASE/a/b/c",
"OUT" : "BASE/x/y/z",
"ARG" : [ "aaa", "bbb", "BASE/a", "BASE=/foo/bar" ]
},
{
"DIR" : "BASE/d/e/f",
"OUT" : "BASE/x/y/z",
"ARG" : [ "ccc", "ddd", "BASE/b", "BASE=/foo/baz" ]
},
{
"foo" : "bar"
}
]
In other words, objects having an "ARG" array, containing a string that starts with "BASE=" should use the string after "BASE=", e.g. "/foo" to substitute other string values that start with "/foo" (except the "BASE=/foo" which should remain unchanged")
I'm not even close to finding a solution myself, and at this point I'm unsure that jq alone will do the job.
With jq:
#!/usr/bin/jq -f
# fix-base.jq
def fix_base:
(.ARG[] | select(startswith("BASE=")) | split("=")[1]) as $base
| .DIR?|="BASE"+ltrimstr($base)
| .OUT?|="BASE"+ltrimstr($base)
| .ARG|=map(if startswith($base) then "BASE"+ltrimstr($base) else . end)
;
map(if .ARG? then fix_base else . end)
You can run it like this:
jq -f fix-base.jq input.json
or make it an executable like this:
chmod +x fix-base.jq
./fix-base.jq input.json
Don't worry, jq alone will do the job:
jq 'def sub_base($base): if (startswith("BASE") | not) then sub($base; "BASE") else . end;
map(if .["ARG"] then ((.ARG[] | select(startswith("BASE=")) | split("=")[1]) as $base
| to_entries
| map(if (.value | type == "string") then .value |= sub_base($base)
else .value |= map(sub_base($base)) end)
| from_entries)
else . end)' input.json
The output:
[
{
"DIR": "BASE/a/b/c",
"OUT": "BASE/x/y/z",
"ARG": [
"aaa",
"bbb",
"BASE/a",
"BASE=/foo/bar"
]
},
{
"DIR": "BASE/d/e/f",
"OUT": "BASE/x/y/z",
"ARG": [
"ccc",
"ddd",
"BASE/b",
"BASE=/foo/baz"
]
},
{
"foo": "bar"
}
]
Some helper functions make the going much easier. The first is generic and worthy perhaps of your standard library:
# Returns the integer index, $i, corresponding to the first element
# at which f is truthy, else null
def indexof(f):
label $out
| foreach .[] as $x (null; .+1;
if ($x|f) then (.-1, break $out) else empty end) // null;
# Change the string $base to BASE using gsub
def munge($base):
if type == "string" and (test("^BASE=")|not) then gsub($base; "BASE")
elif type=="array" then map(munge($base))
elif type=="object" then map_values(munge($base))
else .
end;
And now the easy part:
map(if has("ARG")
then (.ARG|indexof(test("^BASE="))) as $ix
| if $ix
then (.ARG[$ix]|sub("^BASE=";"")) as $base | munge($base)
else . end
else . end )
Some points to note:
You may wish to use sub rather than gsub in munge;
The above solution assumes you want to make the change in all keys, not just "DIR", "OUT", and "ARG"
The above solution allows specifications of BASE that include one or more occurrences of "=".
I am trying to write a perl code to parse multiple JSON messages. The perl code that I have written only parses the values if the JSON file contains only one json message. But it fails when there are multiple messages in that file. It throws error: "Undefined subroutine &Carp::shortmess_heavy". The JSON file is in the following format:
{
"/test/test1/test2/test3/supertest4" : [],
"/test/test1/test2/test3/supertest2" : [
{
"tag1" : "",
"tag2" : true,
"tag3" : [
{
"status" : "TRUE",
"name" : "DEF",
"age" : "28",
"sex" : "f"
},
{
"status" : "FALSE",
"name" : "PQR",
"age" : "39",
"sex" : "f"
}
],
"tag4" : "FAILED",
"tag5" : "/test/test1/test2/test3/supertest2/test02",
"tag6" : ""
}
],
"/test/test1/test2/test3/supertest1" : [
{
"tag1" : "",
"tag2" : false,
"tag3" : [
{
"status" : "TRUE",
"name" : "ABC",
"age" : "21",
"sex" : "m"
},
{
"status" : "FALSE",
"name" : "XYZ",
"age" : "34",
"sex" : "f"
}
],
"tag4" : "PASSED",
"tag5" : "/test/test1/test2/test3/supertest1/test01",
"tag6" : ""
}
],
"/test/test1/test2/test3/supertest6" : []
}
My perl code to parse a single JSON message:
use strict;
use warnings;
use Data::Dumper;
use JSON;
use JSON qw( decode_json );
my $json_file = "tmp1.json";
my $json;
open (my $fh, '<', $json_file) or die "can not open file $json_file";
{ local $/; $json = <$fh>; }
close($fh);
my $decoded = decode_json($json);
print "TAG4 = " . $decoded->{'tag4'} . "\n";
print "TAg5 = " . $decoded->{'tag5'} . "\n";
my #tag3 = #{ $decoded->{'tag3'} };
foreach my $tg3 ( #tag3 ) {
print "Name = ". $tg3->{"name"} . "\n";
print "Status = ". $tg3->{"status"} . "\n";
print "Age = ". $tg3->{"age"} . "\n";
}
To parse multiple JSON objects, use JSON::XS/Cpanel::JSON::XS's incremental parsing.
my $json = '{"foo":"bar"} ["baz"] true 1.23';
my #results = JSON::XS->new->allow_nonref->incr_parse($json);
use Data::Dumper;
print Dumper \#results;
Output:
$VAR1 = [
{
'foo' => 'bar'
},
[
'baz'
],
bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' ),
'1.23'
];
So, i'm making a programm fetching the data as hash from PostgreSQL, which uses a couple JSONs as parameter file and output datafile respectively. And i had some problem with fetching what it shouldn't fetch. Here's parameter json:
{
"queries": [
{
"table_name" : "t1",
"subqueries": [
{
"query_id" : "t1_1",
"query": [
.....some sql query
],
"to_hash" : {
"target_by" : "type_id", // key to index by
"keys" : [
{
"source" : "name", // key in hash from db
"target" : "name" // key in new hash
},
{
"source" : "r",
"target" : "r"
}
]
}
},
{
"query_id" : "t1_2",
"query": [
.....some sql query
],
"to_hash" : {
"target_by" : "type_id",
"keys" : [
{
"source" : "m",
"target" : "m"
}
]
}
}
]
}
]
}
....and here's a perl subroutine:
my $fname = "query_params.json";
my $q_data_raw;
{
local $/;
open(my $fh, "<:encoding(UTF-8)", $fname) or oops("$fname: $!");
$q_data_raw = <$fh>;
close($fh);
}
my $q_data = JSON->new->utf8->decode($q_data_raw);
my %result;
sub blabla {
my $data = shift;
my($tab, $i) = ($data->{table_name}, 0);
if ($data->{subqueries} ne "false"){
my %res_hash;
my #res_arr;
my $q_id;
foreach my $sq (#{$data->{subqueries}}){
my $query = "";
$q_id = $sq->{query_id};
print "\n";
print "$q_id\n";
for(#{$sq->{query}}){
$query .= "$_\n";
}
my $t_by = $sq->{to_hash}{target_by};
my $q_hash = $db_connection->prepare($query);
$q_hash->execute() or die( "Unable to get: " . $db_connection->errstr);
while(my $res = $q_hash->fetchrow_hashref()) {
# print Dumper $res; #print #1
for(#{$sq->{to_hash}->{keys}}){
# print "\nkey:\t" . $_->{target} . "\nvalue:\t".$res->{$_->{source}}; #print #2
$res_hash{$q_id}{$res->{$t_by}}{$_->{target}} = $res->{$_->{source}};
}
$res_hash{$q_id}{$res->{$t_by}}{__id} = $res->{$t_by};
# print Dumper %res_hash; #print #3
}
push #res_arr, $res_hash{$q_id};
# print Dumper $res_hash{$q_id}; #print #4
# print Dumper #res_arr; print #5
$result{$tab}{$q_id} = \#res_arr;
$q_hash->finish();
}
}
}
for (#{$q_data->{queries}}){ // hash from parameter json
blabla($_);
}
my $json = JSON->new->pretty->encode(\%result);
# to json file
....and there's what i get:
{
"t1" : {
"t1_1" : [
{
//type_id_1_* - from first query
"type_id_1_1" : {
"r" : "4746",
"__id" : "type_id_1_1",
"name" : "blablabla"
},
"type_id_1_2" : {
"r" : "7338",
"__id" : "type_id_1_2",
"name" : "nbmnbcxv"
},
....
},
{
//type_id_2_* - from second query
"type_id_2_1" : {
"m" : "6",
"__id" : "type_id_2_1"
},
"type_id_2_2" : {
"m" : "3",
"__id" : "type_id_2_2"
},
............
}
],
"t1_2" : [
{
"type_id_1_1" : {
"r" : "4746",
"__id" : "type_id_1_1",
"name" : "blablabla"
},
"type_id_1_2" : {
"r" : "7338",
"__id" : "type_id_1_2",
"name" : "nbmnbcxv"
},
....
},
{
"type_id_2_1" : {
"m" : "6",
"__id" : "type_id_2_1"
},
"type_id_2_2" : {
"m" : "3",
"__id" : "type_id_2_2"
},
............
}
]
}
}
Somehow, it fetches the queries from other subqueries, what i don't want. And loop seems to be ok, probably. What am i doing wrong?
Well, seems like i initialized #res_arr and #res_hash in wrong level - gotta be in foreach. The output is kinda what i need. At least i don't have duplicates.
Gotta be sleep more X(
....
my $q_id;
foreach my $sq (#{$data->{subqueries}}){
my %res_hash;
my #res_arr;
my $query = "";
$q_id = $sq->{query_id};
print "\n";
print "$q_id\n";
..........
}
I am trying to use jq to "normalize" JSON so that given the input:
[
[
{"M":6, "C":66, "R":0.1},
{"M":6, "C":81, "R":0.9}
],
[
{"M":11, "C":94, "R":0.8},
{"M":11, "C":55, "R":0.46}
]
,
...
]
the output should be:
[
{
"M" : 6,
"X" : [{"66" : 0.1},{"81": 0.9}]
},
{
"M" : 11,
"X" : [{"94" : 0.8},{"55": 0.46}]
},
...
]
I can extract M with map({M: .[0].M but not sure how to proceed
Set X to the result of mapping over the array and creating a one-element object for each entry, with C as the key and R as the value.
map({M: .[0].M, X: map({"\(.C)": .R})})
Since map(f) is defined as [.[]|f], here is an expanded form of Santiago's solution with some additional comments:
[ # compute result array from
.[] # scaning input array
| { M: .[0].M # compute M from the .M of the first element
, X: [ # compute X array from
.[] # scanning each element's objects
| {"\(.C)":.R} # and constructing new {C:R} objects
]
}
]
[foreach .[] as $list ({};
{
"M": $list[0].M,
"X": [foreach $list[] as $item ({};
{"\($item.C)": $item.R}
)]
}
)]