How to process features in FME one by one For Example:- Take all the data of one country add certain data and write in FGDB, then take data of another country and then process it.
If I understand it correctly, what you're looking for is batch processing different files. In that case I think the easiest approach is tu use a WorkspaceRunner in a parent workspace that calls a child workspace that does the processing. In this FME Community example you can find more information on how to build it: https://community.safe.com/s/article/batch-processing-using-the-workspacerunner-1
Related
Our system requirements say that we need to build a slightly unusual producer-consumer processing system. Imagine we have multiple data streams and we take a snapshot each X seconds and put it into the queue for processing. The streams count is not constant. The more clients we have, the more streams we need to process. At the same time, we don't need to process ALL taken snapshots. If we have too many clients and we are not able to process all items in real-time, we would prefer to skip old snapshots and process only the latest ones.
So as I see, the requirements can be met by keeping only one item in a queue for each stream. If there is a new snapshot, while the previous is still there, we need to REPLACE it using stream id as a key.
Is it possible to implement such behavior by Service Bus queue or something similar? Or maybe it makes sense to look into some other solutions like Redis?
So as I see, the requirements can be met by keeping only one item in a
queue for each stream. If there is a new snapshot, while the previous
is still there, we need to REPLACE it using stream id as a key. Is it
possible to implement such behavior by Service Bus queue or something
similar?
To the best of my knowledge, Azure Service Bus does not support this scenario. Through it's duplicate detection functionality, in fact it supports exact opposite of that. You would need to use some other mechanism (like Redis Cache you mentioned) to accomplish this.
I'm using Autodesk Forge to integrate with our remodeling tool. In particular, I need to count objects of different families and types and determine to what room they actually belong. I use Model Derivative API for this purpose. To keep the room/area information I convert .rvt files to .nwc files as suggested here. However, when I retrieve data with GET /modelderivative/v2/designdata/{urn}/metadata/{guid}/properties I face the following problems from time to time:
Room information sometimes disappears from Objects for some reason
Objects disappear from result data for some reason (but they seem to exist when I browse them in A360)
I have no idea, what can be the reason for this.
I have no explanation for the disappearance of room data or objects for you.
If you can provide a reproducible case demonstrating that, I will gladly pass it on to the development team for analysis.
If you are interested in an immediate reliable solution and full control, which I assume is the case, I would suggest following the second bullet item in the advice provided by Eason in the previous answer that you refer to above:
Extract all the room information and object relationships you are interested in via the Revit API, store that data somewhere yourself, and use it later on wherever you like to your heart's content.
Then you will be completely safe and independent of all other components and their unpredictable behaviour.
If the only information that you need is the room containing each family instance, I can even implement a suitable Revit add-in for you.
Another suggestion that might help, if that is indeed the data you require: determine that information in a Revit add-in and attach it to each family instance in your own personal shared parameter. That will ensure that it remains intact through the translation process. Afaik, all shared parameter data is retained, independent of other behaviour.
My real-time document allows the user to edit the file name within the editor (much like Google's own apps). I represent this as a collaborative string so all collaborators see the file renames as soon as possible.
I'm trying to determined the best and most efficient way to keep this collaborative string in sync with the actual file name. There are two scenarios to consider:
In Editor Changes
If a user edits the document name within the editor. In this case we need to use the Drive API to push that change out to the file on Google drive. To avoid race conditions, it is best if only one of the collaborators pushes the change out. The easiest way to do this seems to check if the rename event was local.
I also found it best to add a delay so we are not pushing the rename out to the Drive API with every character change. If a few seconds pass with no more name changes at that point it pushes the change out. This all seems to work well.
External Changes
The harder one and the one I am interested in requesting advice on, the case when the file name is changed externally. For example, if the user renamed the file within the Drive interface itself. We want this change to update our collaborative string to match.
My application is entirely client-side so I can't use webhook push notifications. So my only solution is to poll the file name every X seconds (currently set to 10). But this presents the following problems:
It is API intensive. If you have 4 collaborators that keep the screen open for 8 hour that is 11520 API calls. If my app has lots of users with lots of documents I could see how this might push me past my API limits.
To avoid race conditions (and reduce API calls) we only want one collaborator to check for changes and update the collaborative string if the file name has changed. But how to pick when collaborators might join/exit at any time? Currently I am having each collaborator check anytime the collaborators change if they are the "leader". The "leader" is the collaborator whose session id is the highest. This seems to work but it all seems fairly hackey. Also if collaborators join close together I wonder if it might be possible that a race condition would cause multiple collaborators to think they are the leader.
Is there an easier way? An real-time API function I am missing?
It would be ideal if the real-time API just provided a method that stored the document name. Anytime the real-time API checks for mutations it could grab the latest document name.
I think you've identified the options. There isn't any built in functionality currently to sync it via the Realtime API specifically.
Personally I'd probably back off the poll time a lot.. its probably not critical that the title is always exactly up to date, so asking every few minutes is probably sufficient and would greatly reduce your qps.
In terms of identifying a "leader", I can't think of anything better than something deterministic based on the session id. So long as each rechecks on each session join/leave event, I don't think there should be any issues.
As an example: I'd like to know, when was the Key "tourism" added (or last edited) of this amenities:
http://overpass-turbo.eu/s/1N4
I tried with different APIs but didn't get very far.
Is there a way over any API or do I have to get as SQL Snapshot and somehow look for the last changeset?
You can use the history call of the main OSM API if you know the IDs of the objects wich you want to inspect. The resulting XML includes all versions of the respective object, thus allows one to determine when certain changes were made:
http://www.openstreetmap.org/api/0.6/node/277528308/history
Overpass API supports these kinds of queries since version 0.7.50. Referring to the History API call or extracting that details from a full history planet wouldn't be necessary anymore.
Please refer to this post with more details on diff and adiff keywords.
As the overpass database rebuild is still ongoing, I'll add a working example to this post a bit later. At the moment, June 2014 would be the earliest possible starting date for queries.
I was wondering how does Google Chrome works in regards to its multiprocess architecture. From what I understand there is one process which renders everything and every page has one additional process associated with it. My question is, if a page loads 100MB picture how does it pass it to the renderer process?
In other words, what is the fastest way to pass (copy?) data from one process to another?
In yet another words, if one process produces 100 MB of data, how to let another process to read it? (Note that the data was produces after the process forked).
Edit: If the child process creates the data and the parent process doesn't know in advance the size of the data, how to pass the data from child to parent? I mean, "shared block of memory" has to created by the parent, right? So how much does the parent know how much of space to allocate?
General name for this is IPC - Inter Process Communication.
http://en.wikipedia.org/wiki/Inter-process_communication
Now I do not know how chrome implements it but I hope you get the idea. If I had to choose one I'd say memory sharing or pipe but it could be (almost) any of those.