So I am working on a project and I need to analyse a prefix tree. I was wondering if I can train an MLAI to do so for me, but now I'm stuck on figuring out how to pass a tree as input. The tree can have an arbitrary number of nodes, and the data in each node maybe be a string or some other type. It is also possible that I would need to pass more than just 1 tree as input. I wanna know if what I'm aiming for is even possible or not.
There are varied neural networks that can accept size-varied structured data as inputs, however this is not as simple as taking off the shelf MLP. At the time of writing this answer the most standard set of architectures would be Graph Neural Networks (since trees are just specific graphs).
In the previous reality capture system users could set a parameter which would determine the resolution of the output models. I want to wind up with models about 100-150K vertices. Is there a setting that allows me to request the modeler to keep the number of generated vertices within some bounds, somewhere in the forge API?
The vertex/triangle decimation is usually, what can be called "subjective" task, which can also explain why there are so many optimization algorithms in the wild.
One type of optimization you would need and expect for "organic" models, and totally different one for an architectural building.
The Reality Capture API provides you only with raw Hi-Res results, avoiding "opionated" optimizations. This should be considered just as a step in automation pipeline.
Another step, would be, upon receiving, to automatically optimize the resulted mesh based on set of settings you need.
One of these steps could be Design Automation for 3ds Max, where you feed a model and using the ProOptimizer Modifier within 3ds Max, you output the mesh with needed detail. A sample of this step, can be found here: https://forge-showroom.autodesk.io/post/prooptimizer.
There are also numerous opensource solutions which should help you cover this post-processing step.
I have a hard time understanding how to use vis.js network with a large amount of data generated dynamically. From what I read in the documentation, there are only two easy ways to import data: from gephi or in dot language; right? Isn't that a bit restrictive?
I have no knowledge of gephi or dot language so I decided to use my mysql database which I am used to working with.
So I query my data with php, and generate javascript to build the nodes and edges for the network.
But so far, I only have about 200 nodes and edges (which is like 1/5 of the data I'll have in the end) and it's already very slow to load, it seems like it takes a lot of ressources to display the network (my MacBook Pro gets really loud anytime I open the network page), when vis.js is supposed to be quick and lightweight.
Is that because all the nodes and edges are "written" in the code of the page? Or is it the fact that I use php to query the mysql data?
I don't refuse the idea to work with a json file, or dot language, I just have no idea how to do that... but if it can get me better performances, I'd like to learn how to do it. Can anyone explain in details how it all works? And with either of these methods, can I get different sizes and colors for the nodes and the edges according to the data I need to show (right now I do that in php after querying the data from the database)?
The format required by Vis Network can be serialized and deserialized using const object = JSON.parse(string); and const string = JSON.stringify(object);. There's no need to use Gephi or DOT to simply store data in the data base.
Nodes have size property to change size and both nodes and edges have color to change color. Edges can also inherit color from connected nodes. For more details see the docs for nodes at https://visjs.github.io/vis-network/docs/network/nodes.html and edges at https://visjs.github.io/vis-network/docs/network/edges.html.
Regarding performance there is not much I can tell you without some sample code and data to play with. I tried putting more that 200 nodes to https://thomaash.github.io/me/#/canvas which was built with Vis Network. As I expected it loads instantly and works just fine but I have no idea how fast or slow is MacBook Pro compared to my machine.
I tried to enhance the czml example to move 100, 500 and 1000 objects instead of few by adding loop into the built-in czml code, and the map was stucked after 1000 objects. I saw the lots-of-satellites too, but I think that there are just few hundrends. If cesium doesn't have the means to do that, how can I enhance it to add fast layer of my own ? Is there any way to combine three.js for this enhancement ?
The result has to look something like this.
The short answer is, yes, Cesium can handle 5000 objects. The largest single Cesium app I have personally worked on involved over 35,000 time-dynamic objects.
The full answer is a little more involved. If all you are talking about is Billboard rendering, 5000 is easy. If you want to involve more complex types of visualization, with lots of dynamic geometry and polylines, then it can get a little more complicated. It also depends on the browser and CPU/GPU requirements that you are targeting. Some aspects of Cesium are currently CPU bound, while other things (such as static geometry) are GPU bound. Chrome beats Firefox hands-down in the performance department. Furthermore, it's really easy to write slow JavaScript code, so if you run into problems it's important to use the profiler (the one included with Chrome is great) to pinpoint exactly where the app is spending most of its time (it may not be Cesium).
Cesium developers are always on the lookout to improve performance and there's actually a lot of work being done in the CZML & DynamicScene area right now. If you run into a specific bottleneck that you are having trouble getting past, we'd be happy to help point you in the right direction.
I've always been intrigued by Map Routing, but I've never found any good introductory (or even advanced!) level tutorials on it. Does anybody have any pointers, hints, etc?
Update: I'm primarily looking for pointers as to how a map system is implemented (data structures, algorithms, etc).
Take a look at the open street map project to see how this sort of thing is being tackled in a truely free software project using only user supplied and licensed data and have a wiki containing stuff you might find interesting.
A few years back the guys involved where pretty easy going and answered lots of questions I had so I see no reason why they still aren't a nice bunch.
A* is actually far closer to production mapping algorithms. It requires quite a bit less exploration compared to Dijikstra's original algorithm.
By Map Routing, you mean finding the shortest path along a street network?
Dijkstra shortest-path algorithm is the best known. Wikipedia has not a bad intro: http://en.wikipedia.org/wiki/Dijkstra%27s_algorithm
There's a Java applet here where you can see it in action: http://www.dgp.toronto.edu/people/JamesStewart/270/9798s/Laffra/DijkstraApplet.html and Google you lead you to source code in just about any language.
Any real implementation for generating driving routes will include quite a bit of data on the street network that describes the costs associate with traversing links and nodes—road network hierarchy, average speed, intersection priority, traffic signal linking, banned turns etc.
Barry Brumitt, one of the engineers of Google maps route finding feature, wrote a post on the topic that may be of interest:
The road to better path-finding
11/06/2007 03:47:00 PM
Instead of learning APIs to each map service provider ( like Gmaps, Ymaps api) Its good to learn Mapstraction
"Mapstraction is a library that provides a common API for various javascript mapping APIs"
I would suggest you go to the URL and learn a general API. There is good amount of How-Tos too.
I've yet to find a good tutorial on routing but there are lots of code to read:
There are GPL routing applications that use Openstreetmap data, e.g. Gosmore which works on Windows (+ mobile) and Linux. There are a number of interesting [applications using the same data, but gosmore has some cool uses e.g. interface with websites.
The biggest problem with routing is bad data, and you never get good enough data. So if you want to try it keep your test very local so you can control the data better.
From a conceptual point of view, imagine dropping a stone into a pond and watching the ripples. The routes would represent the pond and the stone your starting position.
Of course the algorithm would have to search some proportion of n^2 paths as the distance n increases. You would take you starting position and check all available paths from that point. Then recursively call for the points at the end of those paths and so on.
You can increase performance, by not double-backing on a path, by not re-checking the routes at a point if it has already been covered and by giving up on paths that are taking too long.
An alternative way is to use the ant pheromone approach, where ants crawl randomly from a start point and leave a scent trail, which builds up the more ants cross over a given path. If you send (enough) ants from both the start point and the end points then eventually the path with the strongest scent will be the shortest. This is because the shortest path will have been visited more times in a given time period, given that the ants walk at a uniform pace.
EDIT # Spikie
As a further explanation of how to implement the pond algorithm - potential data structures needed are highlighted:
You'll need to store the map as a network. This is simply a set of nodes and edges between them. A set of nodes constitute a route. An edge joins two nodes (possibly both the same node), and has an associated cost such as distance or time to traverse the edge. An edge can either either be bi-directional or uni-directional. Probably simplest to just have uni-directional ones and double up for two way travel between nodes (i.e. one edge from A to B and a different one for B to A).
By way of example imagine three railway stations arranged in an equilateral triangle pointing upwards. There are also a further three stations each halfway between them. Edges join all adjacent stations together, the final diagram will have an inverted triangle sitting inside the larger triangle.
Label nodes starting from bottom left, going left to right and up, as A,B,C,D,E,F (F at the top).
Assume the edges can be traversed in either direction. Each edge has a cost of 1 km.
Ok, so we wish to route from the bottom left A to the top station F. There are many possible routes, including those that double back on themselves, e.g. ABCEBDEF.
We have a routine say, NextNode, that accepts a node and a cost and calls itself for each node it can travel to.
Clearly if we let this routine run it will eventually discover all routes, including ones that are potentially infinite in length (eg ABABABAB etc). We stop this from happening by checking against the cost. Whenever we visit a node that hasn't been visited before, we put both the cost and the node we came from against that node. If a node has been visited before we check against the existing cost and if we're cheaper then we update the node and carry on (recursing). If we're more expensive, then we skip the node. If all nodes are skipped then we exit the routine.
If we hit our target node then we exit the routine too.
This way all viable routes are checked, but crucially only those with the lowest cost. By the end of the process each node will have the lowest cost for getting to that node, including our target node.
To get the route we work backwards from our target node. Since we stored the node we came from along with the cost, we just hop backwards building up the route. For our example we would end up with something like:
Node A - (Total) Cost 0 - From Node None
Node B - Cost 1 - From Node A
Node C - Cost 2 - From Node B
Node D - Cost 1 - From Node A
Node E - Cost 2 - From Node D / Cost 2 - From Node B (this is an exception as there is equal cost)
Node F - Cost 2 - From Node D
So the shortest route is ADF.
From my experience of working in this field, A* does the job very well. It is (as mentioned above) faster than Dijkstra's algorithm, but is still simple enough for an ordinarily competent programmer to implement and understand.
Building the route network is the hardest part, but that can be broken down into a series of simple steps: get all the roads; sort the points into order; make groups of identical points on different roads into intersections (nodes); add arcs in both directions where nodes connect (or in one direction only for a one-way road).
The A* algorithm itself is well documented on Wikipedia. The key place to optimise is the selection of the best node from the open list, for which you need a high-performance priority queue. If you're using C++ you can use the STL priority_queue adapter.
Customising the algorithm to route over different parts of the network (e.g., pedestrian, car, public transport, etc.) of favour speed, distance or other criteria is quite easy. You do that by writing filters to control which route segments are available, when building the network, and which weight is assigned to each one.
Another thought occurs to me regarding the cost of each traversal, but would increase the time and processing power required to compute.
Example: There are 3 ways I can take (where I live) to go from point A to B, according to the GoogleMaps. Garmin units offer each of these 3 paths in the Quickest route calculation. After traversing each of these routes many times and averaging (obviously there will be errors depending on the time of day, amount of caffeine etc.), I feel the algorithms could take into account the number of bends in the road for high level of accuracy, e.g. straight road of 1 mile will be quicker than a 1 mile road with sharp bends in it.
Not a practical suggestion but certainly one I use to improve the result set of my daily commute.