I'm building an app which is made of a webserver (currently using NodeJS but doesn't really matter here) and a Chrome extension.
The point of the Chrome extension is to display some data from the server. For this, I need the extension to be aware of changes on the server-side. To do that, I see 3 possible strategies :
1. Websocket
Having a websocket in the background that triggers updates in real time.
2. Regular fetch
But I also can have a timer that will fetch new data on a defined frequency (something like every minute)
3. Event triggered fetch
Another solution would be to detect browser activity, and to trigger a server call during that activity (eg. on active tab changed)
The tradeof I see is browser memory/network usage vs. realtimeness.
To define realtimeness necessity, I'd compare it to an unread email indicator: it doesn't matter if the user is made aware a minute after, but conceptually, it means always displaying out-dated data, which doesn't sound great to me.
On the other hand, if using websockets, or event listeners come at a too high cost and will eventually slow down the browser too much, I'd rather have a minute delay. :)
But maybe fetching the server every minute with most of the time no update to send may be even more expensive than keeping a websocket alive...
Well, any insights on this tradeof and how to choose would be highly appreciated!
Special thanks to #Rob-w for helping me to reformulate the question.
Related
I have been using navigation timing api to collect actual user data about my site performance. Basically, every time a user visits the site, I send this data back to server and store it in the db.
Now that I analyze this data, I am finding that domainLookupStart is abnormally high even when there are no redirects happening. On average, over 3 months of data, its as high as ~400ms. This is making up for majority of my "Time to first byte".
I was wondering if someone has a better understanding of what work happens before the browser fires the domainLookupStart navigation timing event. And is there something I can do to bring this time down?
It can sometimes be the unload event on the previous page. The unload event allow some JavaScript execution before the page is closed, and it's often used by tracking scripts to save the time spent on the page by users. I've even seen a script sending a synchronous ajax requests here!
It could be on your page or on a third party page that drives users to your website.
You can use Chrome DevTool's Performance tab (previously called "Timeline") to record what's going on during unload and maybe you'll find something.
I am developing a Chrome App that needs to work off-line, i.e., I want it to use the App Cache, rather than sending a request to the server, as much as possible. However, should it be necessary to hit the server, then I would like to include a few bytes of data (so the server can collect and analyse some statistics about, for example, how many previous requests were served by the cache alone).
I can not send the data bytes in the URL query, because this defeats the cache (and is a well known technique for ensuring the cache is bypassed - exactly what I do NOT want to do).
I can't use a POST request, as that will also defeat the cache.
I have thought about including data in a header, but am unsure how, and unsure if this will do the trick as everything I have found about this idea recommends against it. I do not want to get in the trap of relying on something completely undocumented in case it stops working in the future (and I am not sure what will work today).
Can I include data in a GET request that will have absolutely no effect on the App Cache (in Chrome in particular), but is available to the server if and when the request makes it that far?
See comment above; this question isn't really about Chrome Apps. It's more about webapps that use App Cache.
It might be helpful to decouple normal web requests from the reporting feature. You are trying to piggyback the stats on the normal requests, but there doesn't seem to be a need to do so. Moreover, for the occasional client that caches perfectly, you'll never hear from that client because the reporting never occurs. So even in the best case the design would have been flaky.
Instead, you could increment a counter every time you make a request that might be cacheable, and keep it somewhere persistent like in localStorage. Then periodically report it to a noncached destination. You can then subtract the traffic you are receiving at the web server (which represents uncached requests) from that reported total (which represents all requests, cached or not). The difference is cached requests.
I've been writing an extension that allows the user to issue voice commands to control their browser, and things were going great until I hit a catastrophic problem. It goes like this:
The speech recognition object is in continuous mode, and whenever the onerror: 'no-speech' or onend events fire, it restarts. This way, the extension is constantly waiting to accept input and reacts whenever a command is issued, even after 5 minutes of silence.
After a few days of of development, today I reached the point where I was testing it in practical use, and I found that after a little while (and with no change to anything on my part), my onend event started firing constantly. As in, looking at the console, I would see 18,000 requests being made in the space of three seconds, all being instantly denied, thus triggering onend and restarting the request.
I'm aware that it would be optimal to wait for sound before sending a request, or to have local speech recognition capabilities without the need for a remote server, but the present API does not allow that.
Are my suspicions correct? Am I getting request limited?
Are my suspicions correct? Am I getting request limited?
Yes
I'm aware that it would be optimal to wait for sound before sending a request, or to have local speech recognition capabilities without the need for a remote server, but the present API does not allow that.
To hide the IP source of your request you can use anonymizer networks like Tor, though it will not be fast.
It's naive to assume Google will spend resources to process all audio being recorded on your system. In your application development it is better to rely on API which provides at least some guarantees. It could be either commercial API or open source implementation like CMUSphinx.
With CMUSphinx, you can also properly implement command keyword detection and increase accuracy by specifying the grammar of the commands.
You could also use a Voice Activity Detection (VAD) algorithm to detect when a user is talking. This can be done by either setting a volume threshold or a frequency threshold (Human speech is usually less than 400hz for example). This way, you won't send useless requests to Google unless those conditions are meant. I would not recommend using Tor as this would significantly increase latency. CMUSphinx is probably the best local system option, but if still want to use a web-based service, I would recommend either using a Voice Activity Detection algorithm or finding a different web-based software.
As far as I know, at the current moment, late 2011 the max-connections-per-server limit remains 6. Please correct me if I am wrong. This is bad that we cannot fix this easily as in Firefox. As far as I know this value is hardcoded.
One of the solutions is to download the Chromium's sources and rebuild them. Is there a more easy solution?
Is there any tricky way to hack this without creating a dozen of mirror-domains?
Why I'm asking the question: My task is to create a html-javascript slideshow that will run inside a fullscreened browser, and a huge monitor is hanging on the wall. The javascript is really complicated, it preloads photos and makes a lot of ajax calls to my web services. If WIFI connection is slow, if 6 photos are loading, the AJAX calls fail, the application runs bad. I want a fast solution based, on http or browser or ubuntu tweak something else, because rebuilding the javascript app will take days.
Offtopic: do you know any other things that can be tweaked in my concrete situation?
IE is even worse with 2 connection per domain limit. But I wouldn't rely on fixing client browsers. Even if you have control over them, browsers like chrome will auto update and a future release might behave differently than you expect. I'd focus on solving the problem within your system design.
Your choices are to:
Load the images in sequence so that only 1 or 2 XHR calls are active at a time (use the success event from the previous image to check if there are more images to download and start the next request).
Use sub-domains like serverA.myphotoserver.com and serverB.myphotoserver.com. Each sub domain will have its own pool for connection limits. This means you could have 2 requests going to 5 different sub-domains if you wanted to. The downfall is that the photos will be cached according to these sub-domains. BTW, these don't need to be "mirror" domains, you can just make additional DNS pointers to the exact same website/server. This means you don't have the headache of administrating many servers, just one server with many DNS records.
I don't know that you can do it in Chrome outside of Windows -- some Googling shows that Chrome (and therefore possibly Chromium) might respond well to a certain registry hack.
However, if you're just looking for a simple solution without modifying your code base, have you considered Firefox? In the about:config you can search for "network.http.max" and there are a few values in there that are definitely worth looking at.
Also, for a device that will not be moving (i.e. it is mounted in a fixed location) you should consider not using Wi-Fi (even a Home-Plug would be a step up as far as latency / stability / dropped connections go).
BTW, HTTP 1/1 specification (RFC2616) suggests no more than 2 connections per server.
Clients that use persistent connections SHOULD limit the number of simultaneous connections that they maintain to a given server. A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy. A proxy SHOULD use up to 2*N connections to another server or proxy, where N is the number of simultaneously active users. These guidelines are intended to improve HTTP response times and avoid congestion.
There doesn't appear to be an external way to hack the behaviour of the executables.
You could modify the Chrome(ium) executables as this information is obviously compiled in. That approach brings a lot of problems with support and automatic upgrades so you probably want to avoid doing that. You also need to understand how to make the changes to the binaries which is not something most people can pick up in a few days.
If you compile your own browser you are creating a support issue for yourself as you are stuck with a specific revision. If you want to get new features and bug fixes you will have to recompile. All of this involves tracking Chrome development for bugs and build breakages - not something that a web developer should have to do.
I'd follow #BenSwayne's advice for now, but it might be worth thinking about doing some of the work outside of the client (the web browser) and putting it in a background process running on the same or different machines. This process can handle many more connections and you are just responsible for getting the data back from it. Since it is local(ish) you'll get results back quickly even with minimal connections.
I have an HTML5/jquery mobile web app at http://app.bluedot.mobi. It is used for long distance races to track competitors via SPOT satellite tracking. The issue I have not yet resolved is that when loading the app when no data connection exists, the browser throws a "no data connection" alert popup as it is attempting to fetch the manifest during the checking event. Even when a data connection is present, loading the app can take a very long time. There are ~ 500 files to check. The fastest way to load the app (from a phone) is to be in airplane mode and dismiss the browser's alert - not so elegant.
Rather than force an update on users who tend to be in the backcountry with a spotty connection, I want to use applicationCache.update() programmatically, giving the user control over the process and speeding up app load whether on or offline.
Is this currently possible with the HTML5 spec and respective browser implementations?
Sounds like you need the abort() method. Unfortunately it is very new, and it will probably be a while before it is implemented by the majority of mobile browsers.
There are ~ 500 files to check.
It sounds like you're implying that the browser checks each file to see if there's any of them which has changed. This is not correct. The browser only checks the manifest file if that has changed, and that is a simple byte check. If the manifest file has not changed, the browser believes nothing has changed.
So if your application is slow to start, it might be your because your application is complex and there's alot of HTML and Javascript to parse. I would advise you to take a look at the application and see if there's anything you can optimize. In that case, you might want to take a look at Yahoo's Best Practices for Speeding Up Your Web Site page.
For example, I noticed you have a lot of Javascript code in the HEAD section. The beforementioned article advices us to move all Javascript (To the extent of what's possible) to the bottom of the page, so that the browser can start rendering the page as soon as possible. And there's a lot of other sound advice in the article. So take a look, I'm sure you'll find it useful. :-)