I am working on a tool that would allow the user query the properties of Revit model elements stored on Autodesk BIM360. I am using Forge indexes API and implement a workflow similar to the one documented here.
It works great for small models, however it starts crashing on large ones. Querying the entire property index with Get Index Properties results in error code 0, while applying any query to the index (Post Queries) returns a query in state = FAILED, with the error as follows:
type: "RuntimeError"
title: "AmazonS3Exception"
detail: "The character number in one record is more than our max threshold, maxCharsPerRecord: 1,048,576"
This seems to be related to the issue explained in this blog post, namely querying the entire json collection as a single object by calling from S3Object[*] as stated in Model Property Service Query Language Reference. Is there a way to fix it on the API caller end or is it a bug on the side of Forge? Help will be more than welcome. Thanks!
Related
I'm working on creating a forge viewer-based web system linking with my client's BIM360 environment. For the model data extraction part, I refered to the examples below.
(1) https://github.com/xiaodongliang/forgeviewer_embed_in_powerbi_report/tree/master/forge-model-properties-excel
(2) https://github.com/Autodesk-Forge/bim360appstore-model.derivative-nodejs-xls.exporter
However, I realized that the dbIds from the Forge Viewer and the sample codes are different. Is this the reason because of the SVF version discrepancies (i.e., SVF1 vs. SVF2)? If yes, any suggestion to resolve it?
Moreover, some models were not correctly processed when I tried to extract model data using the second example code (i.e., ForgeXLS.js example). The code was not processed after calling "prepareTables" function. It seems like it has issues to conduct "getMetadata", "getHierarchy", and "getProperties" functions. Could you let me know some possible reasons?
SVF version discrepancies
yes, the dbId of SVF is not consistent in all versions of the model. SVF2 dbId can be same in different versions, but the ideal is to take advantage of external id of the object. i.e. build a map from the first version of the model with external id. When you want to use dbId, search the dbId by external id
extract model data using the second example code (i.e., ForgeXLS.js example)
I'd believe it is just because the metadata / properties of the model version sis too large. you may try to add the header forceget when fetching the properties
https://forge.autodesk.com/en/docs/model-derivative/v2/reference/http/urn-metadata-guid-properties-GET/
I configured our viewer to use SVF2 format yesterday as per these instructions: https://forge.autodesk.com/en/docs/model-derivative/v2/developers_guide/notes/
I have observed that when selecting a node in the model, we get different Id's from SVF vs SVF2.
I usually pull the metadata and properties using the following GET calls:
https://forge.autodesk.com/en/docs/model-derivative/v2/reference/http/urn-metadata-guid-GET/
https://forge.autodesk.com/en/docs/model-derivative/v2/reference/http/urn-metadata-guid-properties-GET/
It has been mentioned here that the SVF2 Id's are computed differently, is there a way to get the Id's of a SVF2 model without loading it into the viewer first?
Apologizing for any inconvenience had caused. This is a known issue that our engineering team is evaluating, and we're hearing customers' voices. Could you consider providing your use case to SVF2 objectId with more details and the paint points you might have if the objectId between GET properties and viewer are inconsistent? Currently, there are two workflows that we're interested in as the below. Could you kindly point out which one your workflow is?
Normal workflow (GET properties -> Viewer), e.g., get dbIds(objectIds) by filtering properties with some conditions with GET properties, then partial load objects with LMV.
Reverse workflow (Viewer -> GET properties), e.g., select some objects in LMV, then post back to GET properties to get specific properties to store in DB.
Note. If the information you want to share cannot be posted publicly, please send it to forge[DOT]help[AT]autodesk[DOT]com instead. Meanwhile, please remove any confidential information and personal data from anything you want to share with us before sending it out.
Update 2021/07/21
Just to let future devs know that SVF2 is now GA, article is here.
My understanding was that the derivative properties were base on the PropertyDatabase. However it seems in Revit type parameters (as opposed to instance parameters), for example used in a custom family based on shared parameters seem to be handled differently.. They appear in the derivative properties of the {3D} view, but not in the PropertyDatabase. Why aren't they in the PropertyDatabase as well, where else is that information kept?
Whether you're accessing the properties via the GET :urn/metadata/:guid/properties endpoint, through Forge Viewer APIs, or any other way, the data is still the same.
You are correct though that the "inherited properties" are handled slightly differently. For example, in case of Revit documents, certain properties may be defined on the "family type" node, and it's up to the consumer of this data to make these properties visible on "instances" as well. It's possible that the Model Derivative endpoint handles this, including the family metadata in each instance as well.
Btw. we've recently discovered this issue in Forge Viewer's search function as well. When searching for a specific string, the method would only return "family type" objects as matches, but non of their instances. This is currently being addressed by the engineering team.
The model viewer has the ability to get properties by passing a filter: viewer.model.getBulkProperties(dbIds, ['externalId', 'Category'], function) where we can limit the results to just the two properties 'externalId' and 'Category'.
It would be a huge benefit for us to have this same filtering capability from the model derivative api:
https://developer.autodesk.com/en/docs/model-derivative/v2/reference/http/urn-metadata-guid-properties-GET/
We have Revit files with 40,000+ parts, and it can take over 15 minutes to query for properties, but we are getting far more data than we need.
it is a reasonable enhancement. I logged it as an internal ticket DERI-4610.
If you have used Extractor to download the whole SVF dataset to local , you could try with extract the properties from properties.db (the other post tells more). This is a lite sql database which is actually used by Derivative API on Forge cloud. I'd think there is some smart ways to filter the specific properties by the db file.
In this https://developer.gooddata.com/article/data-modeling-api there is a logical data model and its corresponding JSON. However, I can't seem to find out how to extract JSON from a logical data model via the REST API. Is there a way to do this other than using the single load interface (which would be very inefficient)?
For the record, my end goal is to make a tool that extracts that JSON (which would be in dev), then post that to the ldm manager2, and then apply the suggested changes through the returned MaQL to production. Any help is greatly appreciated
Currently this works only for Getting or Updating the entire Project. Anyway you can GET all model definition by simple API call. See the documentation:
http://docs.gooddatadrafts.apiary.io/
There is a GET request which is asynchronous. You can build some logic on the top of that on your end. You can get all models, store per datasets information, but at the end you need to POST the "final version" and all updates will be applied.
Let me know if I can help you with anything!
Regards,
JT