I am writing a Bing Map based Universal App for Windows Phone and Windows 8 that shows some quite large map layers.
Writing the initial app was no problem (the tutorial I followed is at http://blogs.msdn.com/b/rbrundritt/archive/2014/06/24/how-to-make-use-of-maps-in-universal-apps.aspx), however I now am experiencing major problems rendering a layer that contains thousands of polylines, with tens of thousands of co-ordinates.
The data is just too big - on Windows 8.1, the map crashes the application, while on Windows Phone 8.1, the layer takes a very long time to render.
According to http://blogs.msdn.com/b/bingdevcenter/archive/2014/04/23/visualize-large-complex-data-with-local-tile-layers-in-bing-maps-windows-store-apps-c.aspx, I should speed it up by converting it to a local tile layer, however, the program mentioned in the article (MapCruncher) requires a PNG as input. The question is, how do I convert my map data to a PNG? I can have the data as a shapefile, KML file, or a CSV file. Is there another way I should be doing this? I know I can do this via Geoserver, however my app has to have offline support and so cannot download from the web server the appropriate files as needed.
If anyone has an other ways I could approach this speed issue with large layers, then that would be greatly appreciated. I am aware that I can speed up rendering of a layer in Bing Maps via quadtrees, however most of what I have found is theoretical. If anyone has some code I can plug in to this, that would be very helpful.
Local tile layers are fine if you only have data in a small area, or only want to show the data for a few zoom levels. Otherwise the number of tiles grows drastically and will make your app huge. If your data changes regularly, or you want to support all zoom levels of the map you should store your data on a server and expose it as a dynamic tile layer. A dynamic tile layer is a web service that generates a till on demand from your data. You can add caching to the tiles for performance. This is the best way to handle large data sets and one I have used a lot. In fact I have a demo here: http://onsbingmapsdemo.cloudapp.net/ This data set consists of 175,000 complex polygons that equates to about 2GB of data.
I have an old blog post on how to do this here: http://rbrundritt.wordpress.com/2009/11/26/dynamic-tile-layers-in-the-bing-maps-silverlight-control/
If you prefer working with MVC you might find these project useful:
https://ajaxmapdataconnector.codeplex.com/
https://dataconnector.codeplex.com/
Related
I have a fairly simple, and what I would think to be common mapping web mapping project to complete. I'm struggling in my selection of a web mapping API. Thus far, I've not been able to one that meets the following requirements.
Able to display thousands of points in one view without choking crippling the browser. To be specific, I'd say I would like to display roughly 30,000 points at one time and still be able to navigate around a slippy map without degraded performance.
Local maps. The web server will run on the local client, so being able to display a map without reaching out to the internet (even if it's a very basic map) is an absolute requirement.
Render dynamic data from a database onto a map (most API's are meeting this requirement).
Draw polygons directly on the map, and export the lat/lon values of all vertices.
In your experience working with map api's, do any of them meet the requirements above?
I've looked at OpenLayers 3, Leaflet, and Polymaps. Aside from reading every piece of documentation ahead of time, I can't discern if any of these would fill all requirements. Again, I'm hoping someone with experience with any API could point me in the right direction.
Thanks!
As for Leaflet:
Thousands of points: you could use one of these plugins:
Leaflet.markercluster, clusters your points into "groups of points" at low zoom levels.
Leaflet MaskCanvas, replaces all your points by a single canvas layer.
Local maps: as long as you provide a way to create image tiles (even on the local machine), most mapping libraries should work.
Dynamic data: depending on what you call "dynamic", all mapping libraries should provide you with built-in methods to display your data.
Drawing polygons and export lat/lon vertices: use Leaflet.draw plugin
OpenLayers 3 would very probably provide you with all these functionalities as well.
First, I'd like to let it be known that this is my first journey into developing mapping applications with such a large set of data. So, I am in way locked into the techniques or technologies listed below...
The Overview
What I am trying to accomplish is the ability to load my company's market information for the United States onto some form of web-based mapping software. Currently I am trying to accomplish this with Google Maps JS api and GeoJSON data, but am not against other alternatives. There are roughly 163 files (ranging from 6MB to 200KB) that I have exported from ArcGIS into GeoJSON files, and then loaded into a database (currently as GeoJSON strings). We have developed a UI that loads the data based on the current map bounds and Max/Min calculations in the corresponding records.
The problem I'm running into is the render time on the map itself, which is annoying when switching between different regions, states, or zoom levels. The API calls to load the data are acceptable in regards to the size of the data being retrieved. My boss said it is great for a proof of concept, but would like to see it much, much faster.
The Questions
What suggestions could you offer to increase the render time?
Are there better options (3rd party libs, techniques, etc) for what I'm trying to accomplish?
It was suggested by a co-worker to export the map shapes and just use the images for overlaying the information based on the coords. Any input on this?
One solution to your problem could be reducing number of vertices in polygon or polyline layers (files) while preserving feature's shape. In that way geoJSON file would be smaller and easier to render.
You can use tools available in ArcGIS for Desktop.
See
How to Simplify Line or Polygon
or
Simplify Polygon
I'm sure there are similar tools in QGIS or any other open source GIS software.
There are other solutions to your problem like prerendering data in map tiles that could be overlayed over Google Maps basemap(s). That is probably the fastest solution but more complicated.
Maybe this could be good starting point.
Just couple of random thoughts.
On one project I worked on I noticed terrible performance with geoJson I loaded via REST service.
I found out that browser's JavaScript engine had hard time parsing large geoJson responses. I finally decided to load data in batches (multiple requests). Performance increased enormously.
I noticed you use 163 files (layers?). Do you display them at the same time (no no no) or you have some control panel to switch layers on/off? If you display them at the same time then performance will suffer for sure and you should look into generating map tiles with thematically grouped layers. In that way you'd come up with couple of tile layers that you can switch on and off.
Also, if data used to generate tiles changes frequently than tiles aren't good idea. Then maybe you should look into Web Map Services. WMS generates maps on the fly from data stored in database. GeoServer supports WMS and it's easy to configure and use.
I don't have good understanding of Big data libraries/spatial data. Can someone help to demystify how google maps, bing maps (any maps) store data internally? If I want a Image at some latitude 'x' and some longitude 'y' at some zoom factor, how is this data stored for efficient retrieval? Also, there is quite a bit of information associated at every point in cell, how is this data stored? Links to relevant pages or blogs will be of great help.
Thanks
Most highly scalable map providers start out with the spatial data stored in a spatial database, including road lines, rail lines, polygonal boundaries, hydrography, etc. However, they don't render the maps directly from that data for every user request because that would take far too long for the millions of requests they get every day.
Instead, tiles of the maps are rendered offline using the spatial data, and stored as images on disk so they can be retrieved quickly. The tiles are regenerated for different zoom levels so that more detail shows up at different levels. Since lower zoom levels may have millions of tiles, but are rarely visited, those may be generated at viewing time, instead of being pre-generated.
Then they set up their applications (Javascript-based or otherwise), to request all the tiles over a given viewing area and zoom level, and piece the tiles together to generate a complete map.
WMTS is one open standard for delivering map tiles over the web, although WMTS doesn't define the process for generating tiles. GeoWebCache is a service which, paired with a WMS server like GeoServer or MapServer, can be used for generating tiles from spatial data.
You can check out elasticsearch. Its an open source search engine extensively used for bigdata technologies. You can perfrom geospatial queries on top of huge data in elasticsearch with high efficiency.
Is there any general advice when to use KML layers or the Javascript API? There are several needs for the app. I need to display markers for moving object on the map. These will be refreshed every minute in average. But there will be a lot of static markers to be displayed on demand. These are just markers. But in another screen I need to draw the driven routes.
Actually I'm not sure what's the best way of displaying all the needed data. KML? Only get the data and to the rest by the API on the fly? Generate the code on the server and just upload it to the browser. What would be the best in terms of performance and and easy usage for the developer?
I think javascript layers are faster than kml.
(The fastest speed is to generate image tiles - but that takes a lot of time and needs to be done statically).
I'd use AJAX to generate the data on the server and send it to the API on the fly.
I think KML is useful if you want to share the data with other websites and people using desktop GIS software. It is extremely easy to use someone else's KML layer. It is much harder to use their javascript array.
I want to make map of a particular region, just for education purpose.
I have gone through several web site , which tell to make maps.
like
http://worldwind.arc.nasa.gov/java/demos/,
But i did't find the way as GOOGLE/BING/YAHOO show their maps like its graphics quality and all that its features.
i need to add features in the map like "adding a tag, show distance", as google show.
I would also like to connect it to database(My own).
I don't want to use google api, because i can't connect it with my own data base.
I went through this before for me it was a 4 tiers application:
Database, I used Sql Server 2008 Spatial Database.
Map Manager application, its main objective is to import the spatial data from different sources into the sql database, and other minor objectives like (styling the map objects, edit visible zoom level of objects, ...)
WMS, a RESTful wcf service that renders the tiles.
Map Client Control, a silverlight control that renders the tiles from the WMS.
You can find a lot of resources on the web on how to manipulate the spatial data (read, write, search...) and for rendering tiles I used the Bing Maps Tile System
http://msdn.microsoft.com/en-us/library/bb259689.aspx
And what remains is just the implementation,
The Map Client Control requests a tile from the WMS.
The WMS calculates the tile boundaries and retrieves the spatial data within these boundaries and render them on an image and returns it to the client.
The Map Client Control is responsible for layouting the tiles, zooming, panning...
You can cache the rendered tiles by implementing some cache techniques.
And finally I wish you good luck.