I’m working on a digital twin projects and I’m facing an issue that is crashing my occulus app.
I have an API call that contains information around the color for around 1000 objects that I want to update every couple of seconds.
Basically I want for unreal to use a VARest plugin and fetch the RGB value and update my material.
I seem to be able to run it in my editor correctly. When I compile it for the Meta Quest 2 I keep getting
Maximum number of UObjects (196608) exceeded, make sure you update MaxObjectsInGame/MaxObjectsInEditor/MaxObjectsInProgram in project settings.
I think the issue lies with the fact that when reading the API I’m running a for each loop that is creating an objects. So when I run obj list i can see that there is a lot of objects in VaRestJsonValue 2,421,102
Related
I'm finalizing a Data Studio connector and noticing some odd behavior with the number of API calls.
Where I'm expecting to see a single API call, I'm seeing multiple calls.
In my apps script I'm keeping a simple tally which increments by 1 every url fetch and that is giving me the correct number I expect to see with getData().
However, in my API monitoring logs (using Runscope) I'm seeing multiple API requests for the same endpoint, and varying numbers for different endpoints in a single getData() call (they should all be the same). E.g.
I can't post the code here (client project) but it's substantially the same framework as the Data Connector code on Google's docs. I have caching and backoff implemented.
Looking for any ideas or if anyone has experienced something similar?
Thanks
Per the this reference, GDS will also perform semantic type detection if you aren't explicitly defining this property for your fields. If the query is semantic type detection, the request will feature sampleExtraction: true
When Data Studio executes the getData function of a community connector for the purpose of semantic detection, the incoming request will contain a sampleExtraction property which will be set to true.
If the GDS report includes multiple widgets with different dimensions/metrics configuration then GDS might fire multiple getData calls for each of them.
Kind of a late answer but this might help others who are facing the same problem.
The widgets / search filters attached to a graph issue getData calls of their own. If your custom adapter is built to retrieve data via API calls from third party services, data which is agnostic to the request.fields property sent forward by GDS => then these API calls are multiplied by N+1 (where N = the amout of widgets / search filters your report is implementing).
I could not find an official solution for this either, so I invented a workaround using cache.
The graph's request for getData (typically requesting more fields than the Search Filters) will be the only one allowed to query the API Endpoint. Before starting to do so it will store a key in the cache "cache_{hashOfReportParameters}_building" => true.
if (enableCache) {
cache.putString("cache_{hashOfReportParameters}_building", 'true');
Logger.log("Cache is being built...");
}
It will retrieve API responses, paginating in a look, and buffer the results.
Once it finished it will delete the cache key "cache_{hashOfReportParameters}building", and will cache the final merged results it buffered so far inside "cache{hashOfReportParameters}_final".
When it comes to filters, they also invoke: getData but typically with only up to 3 requested fields. First thing we want to do is make sure they cannot start executing prior to the primary getData call... so we add a little bit of a delay for things that might be the search filters / widgets that are after the same data set:
if (enableCache) {
var countRequestedFields = requestedFields.asArray().length;
Logger.log("Total Requested fields: " + countRequestedFields);
if (countRequestedFields <= 3) {
Logger.log('This seams to be a search filters.');
Utilities.sleep(1000);
}
}
After that we compute a hash on all of the moving parts of the report (date range, plus all of the other parameters you have set up that could influence the data retrieved form your API endpoints):
Now the best part, as long as the main graph is still building the cache, we make these getData calls wait:
while (cache.getString('cache_{hashOfReportParameters}_building') === 'true') {
Logger.log('A similar request is already executing, please wait...');
Utilities.sleep(2000);
}
After this loop we attempt to retrieve the contents of "cache_{hashOfReportParameters}_final" -- and in case we fail, its always a good idea to have a backup plan - which would be to allow it to traverse the API again. We have encountered ~ 2% error rate retrieving data we cached...
With the cached result (or buffered API responses), you just transform your response as per the schema GDS needs (which differs between graphs and filters).
As you start implementing this, you`ll notice yet another problem... Google Cache is limited to max 100KB per key. There is however no limit on the amount of keys you can cache... and fortunately others have encountered similar needs in the past and have come up with a smart solution of splitting up one big chunk you need cached into multiple cache keys, and gluing them back together into one object when retrieving is necessary.
See: https://github.com/lwbuck01/GASs/blob/b5885e34335d531e00f8d45be4205980d91d976a/EnhancedCacheService/EnhancedCache.gs
I cannot share the final solution we have implemented with you as it is too specific to a client - but I hope that this will at least give you a good idea on how to approach the problem.
Caching the full API result is a good idea in general to avoid round trips and server load for no good reason if near-realtime is good enough for your needs.
I'd like to have a list of just the current titles for all questions in one of the smaller (less than 10,000 questions) stackexchange site. I tried the interactive utility here: https://api.stackexchange.com/docs/questions and it both reports the result as a json at the bottom, and produces the requesting url at the top. For example:
https://api.stackexchange.com/2.2/questions?order=desc&sort=activity&tagged=apples&site=cooking
returns this JSON in my browser:
{"items":[{"tags":["apples","crumble"],"owner":{ ...
...
...],"has_more":true,"quota_max":300,"quota_remaining":252}
What is quota? It was 10,000 on one search on one site, but suddenly it's only 300 here.
I won't be doing this very often, what I'd like is the quickest way to edit that (or similar of course) url so I can get a list of all of the titles on a small site. I don't understand how to use paging, and I don't need any of the other fields. I don't care if I get them, but I'm thinking if I exclude them I can have more at once.
If I need to script it, python (2.7) is my preferred (only) language.
quota_max is the number of requests your application is allowed per day. 300 is the default for an unregistered application. This used to be mentioned directly on the page describing throttles, but seems to have been removed. Here is historical information describing the default.
To increase this to 10,000, you need to register an application and then authenticate by passing an access token in your script.
To get all titles on a site, you can use a Python library to help:
StackAPI. The answer below will use this library. DISCLAIMER: I wrote this library
Py-StackExchange
SEAPI
StackPy
Assuming you have registered your application and authenticated we can proceed.
First, install StackAPI (documentation):
pip install stackapi
This code will then grab the 10,000 most recent questions (max_pages * page_size) for the site hardwarerecs. Each page costs you one API hit, so the more items per page, the few API calls.
from stackapi import StackAPI
SITE = StackAPI('hardwarerecs')
SITE.page_size = 100
SITE.max_pages = 100
# Filter to only get question title and link
filter = '!BHMIbze0EQ*ved8LyoO6rNjkuLgHPR'
questions = SITE.fetch('questions', filter=filter)
In the questions variable is a dictionary that looks very similar to the API output, except that the library did all the paging for you. Your data is in questions['data'] and, in this case, contains a list of dictionaries that look like this:
[
...
{u'link': u'http://hardwarerecs.stackexchange.com/questions/29/sound-board-to-replace-a-gl2200-in-a-house-of-worship-foh-setting',
u'title': u'Sound board to replace a GL2200 in a house-of-worship FOH setting?'},
{ u'link': u'http://hardwarerecs.stackexchange.com/questions/31/passive-gps-tracker-logger',
u'title': u'Passive GPS tracker/logger'}
...
]
This result set is limited to only the title and the link because of the filter we applied. You can find the appropriate filter by adjusting what fields you want in the web UI and copying the filter field.
The hardwarerecs parameter that is passed when creating the SITE parameter is the first part of the site's domain URL. Alternatively, you can find it by looking at the api_site_parameter for your site when looking at the /sites end point.
I am working on a fairly simple Heat Map application where the longitude and latitude of the points will be stored in a SQL Server database. I have been looking at an example that uses an array of objects as follows (eliminated a lot of data for brevity):
/* Sample data to demonstrate Bing Maps Heatmap */
/* http://alastair.wordpress.com */
var CrimeData = [
new Microsoft.Maps.Location(52.67280, 0.94392),
new Microsoft.Maps.Location(52.62423, 1.29493),
new Microsoft.Maps.Location(52.62187, 1.29080),
new Microsoft.Maps.Location(52.58962, 1.72228),
new Microsoft.Maps.Location(52.69915, 0.24332),
new Microsoft.Maps.Location(52.51161, 0.99350),
new Microsoft.Maps.Location(52.59573, 1.17067),
new Microsoft.Maps.Location(52.94351, 0.49153),
new Microsoft.Maps.Location(52.64585, 1.73145),
new Microsoft.Maps.Location(52.75424, 1.30079),
new Microsoft.Maps.Location(52.63566, 1.27176),
new Microsoft.Maps.Location(52.63882, 1.23121)
];
What I want to do is present the user with a list of some sort that displays all the data sets that exist in the database (they each have a name associated with them) and then allow the user to check all or only a select few. I will then need to generate an array like the above to create the heat map. Any ideas on a good approach to this?
What you trying to achieve is more related to a web developement rather than only related to Bing Maps.
To summarize, you have multiple ways to do this but it really depends on what you are capable to do and what you need in the interface.
What process/technology?
First, you need to determine what process you want to follow to display the data and it will set the technology that you will use. The questions that you need to ask yourself are:
Do you want to be able to change the data sets dynamically without refreshing the whole page?
If yes, it means that you will have to use asynchronous data loading through a dedicated web service in order to avoid loading all the information at the initial load of the page.
Do you have lots of data to load?
If so, it might comfort you with asynchronous loading to avoid loading all data.
If not loading every elements in multiple arrays might be the simplest solution.
Implementation
So now, you want to create a web service to load the data asynchronously, you can take a look at the following websites :
http://www.asp.net/get-started
http://www.stefanprodan.com/2011/04/async-operations-with-jquery-ajax-and-asp-net-mvc/
There might be interesting other website, you will be able to find them. If needed, add comment and I'm sure the community will help you.
If you want to generate the data directly in the script, it could be simple as you can compose the JavaScript directly in your dynamically created HTML page (in your ASP.Net markup code or whatever technology you're using).
I have an air (4.5.1) mobile project that send an ArrayCollection to the server (Tomcat/BlazeDS)
The server manage the object and return a string containing the result (ok/error/etc)..
Everything worked fine, until:
I tried to send an ArrayCollection with length > 35000 (not sure border limit).
After sending the arraycollection the UI seems like frozen for a little time, and after that
I got a FaultEvent Error
NetConnection.Call.Failed: HTTP: Failed
The server however received the request, parsed it and returned the result string
So, because the program get the faultevent, I cannot be sure (from the client) that the request is finished correctly...
How can I fix it? and is this problem generated by the length of the arraycollection?
Other ideas?
Thanks
This is an on going issue with Flex/Air/Flash. The problem you are running into is a defualt value for requestTimeout of 30 seconds. Even if you change the value in your remoteObject, it is not getting used correctly. There are MANY MANY documented bugs on adobe regarding this issue. Below is a link to a site that has collected some info about this problem from around the web. To date adobe has yet to fix the problem even though that claim they have in previous versions.
RemoteObject Issue
Hi, i have been having some problems using JSON data in flash builder lately and I was hoping someone could help me out here.
I have spent the past month working solidly on this issue, so I HAVE been looking around, HAVE been trying out everything I can find or think of. I am simply stuck.
I have been working on a flex mobile application for the Blackberry Playbook tablet with Adobe Flash Builder 4.6. It is a reddit app, designed to give users the main reddit feed, subreddits, search function, hopefully log in stuff etc. Of course I need the aid of the reddit API to access this information, of which the documentation can be found here: https://github.com/reddit/reddit/wiki/API/ The api uses either XML or JSON formatted data.
Now onto my problem- Like mentioned above, i want to display the reddit feed inside the app. I want to be able to use a item renderer to customize the data that is shown within each entry of the list.
One entry would consist of:
1) a thumbnail of the image in the post
2) The title of the post
3) a 'like/dislike' button, but this is unimportant at the moment.
Of course to start out, i placed a spark List component onto the design view. Then i configured a new HTTP data service using the Data/Services panel. I specified http://www.reddit.com/r/all.json for the url. I configured the return type, and the did a Test Operation. Everything connected just fine. All the data came through as normal. I will give you an idea of what the data comes back as so that you may understand my issue later on.
Test Operation Results (json data structure):
NoName1
data
after
before
children
[0]
data
media_embed
score
id
title
thumbnail
url
(etc etc...)
kind
[1]
data
media_embed
score
title
thumbnail
(etc etc...)
kind
[2] (array continues)
modhash
kind
As you can see, to get to the thumnail for example, you would have to go through data.children[].data.thumnail. When I tried to bind this data to the spark List component, I specified the data service to be from the one above. Then I specified the Data provider option to be Children[], as this value is typically set to the array. Now this is where the trouble begins. The final option, Label Field, only gave me one value to choose from: 'kind'. So as you can tell, it wasnt expecting the data to go any further nested. It stops at the two value just within each array item, which would be Data and Kind, though it only offers me Kind. I need to go one level further to access Title and Thumbnail. This is my problem.
Now, I have analyzed the code for the binding, and I have tried altering it to accomodate the further nested value. No success what so ever. The following is the code that the binding generates:
<s:List
id="myList" width="100%" height="100%" change="myList_changeHandler(event)"
creationComplete="myList_creationCompleteHandler(event)" labelField="kind">
<s:AsyncListViewlist="{TypeUtility.convertToCollectionredditFeedJSONResult.lastResult.data.children)}"/>
<s:List>
Obviously i would want to have something like along the lines of:
"TypeUtility.convertToCollection(redditFeedJSONResult.lastResult.data.children.data" and then set the labelField="title" or "thumbnail".
I certainly hope someone can help me with this. I am out of my mind as to what to do. If you need any further clarification I would be happy to provide it. I tried to explain the situation above as clearly as possible. Thank you so much.
Ted
I have this situation often: get an XML or JSON data from the server, then trying to use it as a dataProvider for a spark.components.List or for a mx.controls.Menu and then they just wouldn't display the data as I want them, because something in the data is different from what they expect. And then they display wrong XML-children or [Object,Object,etc.]
And in such cases I just create an empty ArrayCollection() which serves as dataProvider instead (and can be sorted and/or filtered too). And when data comes from the server, I push() new Objects into it:
[Bindable]
private var _data:ArrayCollection = new ArrayCollection();
public function update(xlist:XMLList):void {
_data.length = 0;
for each (var xml:XML in xlist)
_data.push({label: xml, event: xml.#event});
}
This always works. And if you get your next problem - flickering of the List, then that is solvable by merging data too.
Good luck with your Playbook development, which is a cool piece of hardware :-)