How are GeoTIFFs persisted in GeoMesa? - geomesa

How are GeoTIFFs saved in GeoMesa and Accumulo given that it is a key-value datastore? Moreover, which indices are added (if!) and on which fields?
Likewise, for other formats such as:
GeoJSON
GML
KML

GeoMesa's raster support is targeted a narrow set of use cases. The documentation for the raster support (http://www.geomesa.org/documentation/user/accumulo/raster.html) has some of the details for ingest and setting up GeoServer. Raster data does need to be in longitude/latitude (and maybe pre-tiled).
GeoMesa's raster data is indexed by spatial extent solely.
(It may be worth looking at more general raster toolkits like GeoTrellis for Hadoop/Spark-based raster support.)
For vector data, GeoMesa has support via its converter library (http://www.geomesa.org/documentation/user/convert/index.html) for ingesting XML, Json, and other file formats. More generally, GeoMesa implements the GeoTools DataStore API. For any format with a GeoTools DataStore implementation, one could read from that source and write to GeoMesa using straightforward Java/Scala code.
GeoMesa's GeoJson support is documented here (http://www.geomesa.org/documentation/user/geojson.html). For GML, I think the converter library might be a sane approach. For KML, it appears there is an unsupported datastore here (https://github.com/geotools/geotools/blob/master/modules/unsupported/kml/src/main/java/org/geotools/data/kml/KMLDataStore.java).

Related

Which APP or software can revert IFC to JSON

Just a little question. I mean, JSON is so good in the Web developing. So I try to ask you which APP or software can revert IFC to JSON.
IFC is a schema built on top of STEP (ISO 10303) format, so thats what would need to be converted, not the IFC aspect. It would be quite trivial to convert STEP syntax to JSON, in fact there is something called IFCxml (https://en.wikipedia.org/wiki/Industry_Foundation_Classes#IFC/ifcXML_specifications. ISO 10303-28) which is IFC using XML syntax instead of STEP syntax any way (JSON and XML being somewhat similar in structure, there are many XML -> JSON converters out there).
However there would be no point as this would not change the fact that the IFC still needs to be processed (not just read) by a client with an understanding of the schema. In this regard it makes no difference what syntax is used as the power of the IFC comes from the higher level heirachy of the schema, not the syntax that is used.
It would probably have the same problems as XML for IFC, i.e huge files in comparison to STEP so probably wouldn't be popular, even if there were tools which could process the IFC entities in JSON format.
OK. Now I find it is a new data format named as ifcjson can deal with it.
It is a new schema, simpler than IFC-SPF and IFCXML.
So maybe we need not a convertor, a new format is also a good choice.

GeoMesa: saving raster data

How can I save in GeoMesa raster data (GeoTIFF, DEM)? Can you please provide code examples? Also, please provide info (links) to the serialization of the raster data in Accumulo
I have only found the following commands line tool:
http://www.geomesa.org/documentation/user/accumulo/raster.html#ingest
http://www.geomesa.org/documentation/user/accumulo/commandline_tools.html#accumulo-tools-raster
http://www.geomesa.org/documentation/tutorials/geomesa-raster.html
Moreover, can I read that the image should be an image pyramid EPSG:4326 in order to ingest it. What will happen if it is not? Will I not be able to ingest it or the image will not be available in multiple zoom levels?
Since the GeoMesa raster module isn't heavily used, sadly there are few examples around writing data to Accumulo using it. That said, I believe you are looking for the 'putRaster' method here [1].
I believe the data must be pre-tiled to be ingested. If it is not pyramided, then only the one zoom level will be ingested. The benefit of the pyramid is for quicker downsampling; if that's not a concern, then no worries!
https://github.com/locationtech/geomesa/blob/master/geomesa-accumulo/geomesa-accumulo-raster/src/main/scala/org/locationtech/geomesa/raster/data/AccumuloRasterStore.scala#L193

Loading json file into titan graph database

I have given a task to load a json file into titandb with dynamodb as back end.Is there any java tutorial or if possible please upload java sample coding...
thanks.
Titan is an abstraction layer so whether you use Cassandra, dynamo, hbase, etc, you merely need to find Titan data loading instructions. They are a bit dated but you might want to start with these blog posts:
http://thinkaurelius.com/2014/05/29/powers-of-ten-part-i/
http://thinkaurelius.com/2014/06/02/powers-of-ten-part-ii/
The code examples work with an older version of Titan (the schema portion) but the concepts still apply.
You will find that the strategy for data loading with Titan has a lot to do with the size of your graph. You said you are loading "a JSON file" so I imagine you have a smaller graph in the millions of edges. In this case, a simple groovy script will likely suffice. Write a script to parse your JSON and write the data to the Titan.

What I benefit from using Geo Json on mobile side, when all the geo processing is done on server side?

I have a mobile app heavily associated with maps, GPS, location etc.
Currently I'm sending real time location data to server as regular Json (Strings...), and in the server I'm converting it to my custom geo objects, querying geo spatial database (PostGis etc.). Then backwards: I'm converting the results to strings and send them back to client as regular Json. Usually the client needs this results to display them to the user, so it's convenient that they are already strings.
My question is what can I benefit from working with Geo Json format instead? It's clear that it enables "offline mode" using SpatiaLite etc. But what else? What about all the conversions I'm using from/to strings of geo locations? Currently I have some very simple custom geo objects (location object etc.). Can I benefit here? Note that in the end it's good for me have the results as strings for the user, and for now offline mode irrelevant.
Is there a rule of thumb that if you are location/map app - it's best practise to use geo Json and not regular Json?
Thanks,
The benefit of using geojson instead of pushing out values as string is that geojson is a documented, specified format. Client-libraries (like d3.js) can make use of it and make transformations, projections and other geo-related operations on the client.
If the format benefits you in your current project depends on the libraries and framworks you use with your client. If for example your Project only needs geo-xml or kml output, geojson will be of no use for you.

Performance Advantages in Storing Documents as JSON in MarkLogic 6

If I were to store the same markup in 2 separate documents, one XML, the other JSON, in MarkLogic 6, does MarkLogic automatically convert the JSON equivalent to XML, and index it in that regard, or are both stored in their respective formats?
What I'm getting at is, does MarkLogic store ALL documents as XML, regardless, and simply apply JSON transformations to JSON documents when queried?
If documents are stored in native format, is there any advantage, in terms of performance, to storing documents in JSON over XML?
Below is an example code-snippet:
if($outputFormat="json") then (: result in json format :)
let $custom-config :=
let $config := json:config("custom")
return (map:put($config, "array-element-names",(xs:QName("lp:lesson_plan"),
xs:QName("lp:instructional_segment"),
xs:QName("lp:strand_type"),
xs:QName("lp:resource"),
xs:QName("lp:level"),
xs:QName("lp:discipline"),
xs:QName("lp:language"),
xs:QName("lp:program"),
xs:QName("lp:grade"),
xs:QName("res:strand_type"),
xs:QName("res:resource"),
xs:QName("res:ISBN"),
xs:QName("res:level"),
xs:QName("res:standard"),
xs:QName("res:secondaryURL"),
xs:QName("res:grade"),
xs:QName("res:keyword"))),
map:put($config, "whitespace","ignore"),
map:put($config, "text-value","value"),
$config)
return json:transform-to-json($finalResult, $custom-config)
else (: finalResult in xml format :)
$finalResult
MarkLogic is XML-native and does need to convert JSON to XML to store it in the database. There is a high-level JSON library to perform transformations. The main functions are json:transform-to-json and json:transform-from-json, and when configured correctly should provide lossless conversions.
I think the main difference from your example is whether you want to convert to XML using your own process or use MarkLogic's toolkit.
For more detailed information, see MarkLogic's docs:
http://docs.marklogic.com/guide/app-dev/json
On disk, MarkLogic stores highly compressed C++ data structures that represent hierarchical trees and corresponding indexes. (OK, that’s an over-simplification, but illustrative nonetheless.) There are two places where you as a developer will typically interact with those data structures: 1) building queries and application logic 2) deserializing/serializing data into and out of this internal data model. Today, MarkLogic uses the XML data model (XDM) for the latter and, correspondingly, XQuery, XPath, and XSLT for the former. We chose this stack for several reasons: XML is good at representing both text mark-up as well as data structures and the tooling around XML is mature and widespread.
Having said that, JSON has emerged as a popular serialization of hierarchical data structures—the “X” in AJAX. While we don't have the same watertight abstraction between JSON and MarkLogic’s internal data model today, we do provide a set of tools that allow you to efficiently and losslessly convert between JSON and the XML data model. Additionally, our REST and Java APIs allow you to store, retrieve, and even query tree structures that originated as JSON without having to think about this conversion step; the APIs handle this in the plumbing.
As for performance, there will be a little overhead converting between a JSON and XDM representation. However, I’d expect that to be negligible for most applications. The real benefits of XML will be in the expressiveness of XQuery, XPath, and XSLT in working with the data. There is no widespread equivalent to these in the JSON world today.
One footnote: The REST API (and thus the Java API wrapper around the REST API) provide a facade for the JSON conversion to XML -- that is, the APIs do the conversion to XML for you.
Usually, you don't need to think about the conversion except when you are creating range and geospatial indexes over the converted elements.
If you need to support JSON documents in your client, then the facade is convenient.
On the other hand, expressing the structure as JSON has no advantages for database operations and some limitations. (For instance, XML has the standards-based, baked atomic data types, schema validation, and server processing with XQuery or XSLT.) So, if you have complete control over the data structure, you might want to write it to the server as XML.
As of MarkLogic 8 (February 2015), JSON is now a native data type, just like XML. This eliminates the needs for a translation layer for applications that want to work exclusively in JSON. In addition, we’ve added JavaScript as a first-class language in the database itself (using Google’s V8 engine). This means that you can write stored procedures, triggers, and even full HTTP applications with JavaScript that runs in the database, close to the data.