On one of my Beanstalk application Cloudfront redirects user to the origin, instead of caching.
Similar to AWS CloudFront with multiple S3 origins, or https://github.com/Wolox/tech-guides/blob/master/infrastructure/docs/aws/docs/cloudfront-eb-configuration.md.
I saw "The most important step is choosing Forward All, cache based on all in the Query String Forwarding and Caching option. When using an EB load balancer as origin, leaving this option on its None option will cause the distribution to redirect requests instead of caching them." suggestion, but Cloudfront behaviour page was updated, and I couldn't find relevant options anymore.
OK, it was simple - Cloudfront has to forward the Host header. I'm curious why it's not a default cache behavior.
From https://aws.amazon.com/premiumsupport/knowledge-center/configure-cloudfront-to-forward-headers/:
Under Cache key and origin requests, confirm that Legacy cache settings is selected. If it's not selected, then follow the steps in the preceding section to create a cache policy. If Legacy cache settings is selected, then complete the following:
For Headers, select Include the following headers.
From the Add header dropdown list, select Host.
Related
Im checking CDN hit/miss from cache, using same PC, same client address and requesting same URL:
***Scenario 1 Cache-Control 1day, 1month, 1 year:
-Hour 12:00
user1 Request URL it not found, cache filled.
-Hour 12:05
user1 Request URL found, cache HIT response.
-Hour 12:10
user1 Request URL not found, cache filled.
***Scenario 2 (using same internet gateway) Cache-Control 1day, 1month, 1 year
-Hour 12:00
At Building Organization User1 Request URL, Url not found but on second request cache hit
-Hour 12:01
At same Building Organization User2 Request same URL, and voila again Url not found but on second request cache hit
***Scenario 3 (using same internet gateway) Cache-Control 1day, 1month, 1 year
-Hour 12:00
At Building Organization User1 Request URL using Edge Browser, Url not found but on second request cache hit then same user on same PC, open Chrome or Firefox Request URL and voila again Url not found and cache again need to fill
Why cache out very soon even if cache-control set for 1day, 1month or 1year, or if use diferent browser? this is a bug?.
It's not a bug. In many metropolitan areas, Google Cloud CDN operates multiple caches. If you check the logs for the cache misses in your example, you will likely find the requests were served by distinct caches. You won't get cache hits from a particular cache until that cache has had a chance to cache the content.
cloud.google.com/cdn/docs/logging describes how to view log entries. In each log entry, the cacheId field identifies which cache served the response. Even once a response is cached, max-age and s-maxage specify only the maximum amount of time that response can be used. There's more information about expiration and eviction at cloud.google.com/cdn/docs/overview#eviction_and_expiration.
Cache modes 1 controls the factors that determine whether and how CDN caches your content. For example if you are using USE_ORIGIN_HEADERS as the cache mode then we should be looking at values mentioned against max-age and s-maxage mentioned in the response to check the TTL of the cached content. Since s-maxage overrides max-age, we shall look at the configured value for s-maxage. If we look at the best practices, it is advised to keep this value a bit large so that the content in the cache does not expire soon.
Also to maximise performance of Google CDNs, we need to increase the amount of incoming requests per url.
Now lets consider an example where a user is using a HTTPS load balancer with Cloud run endpoints in europe-west1. This means that request to certain URL can go to either endpoints in either zone: europe-west1-a/b/c. The request first reaches primary GFEs near to the user and then the secondary GFEs available in each zone if primary GFEs do not have the requested data in their cache.
Now a new request will hit the primary GFEs nearer to the user and the data will get cached in that GFEs cache after contacting the backend, provided no data related to the request was found in that GFEs cache. Now there is a high possibility that primary GFE used for the first request might not be used again for the second request. For data to be served from primary GFE's cache, the request should have entered all the primary GFEs nearer to the user. Scenarios wherein the data was not present in primary GFEs cache, the request goes to secondary GFEs available in the region. Let us say the second request went to secondary GFE and that did not have any data pertaining to that URL in its cache. In that case the request will go the backend. Now considering for the third request, a primary GFE chose another secondary GFE within a zone (not the one mentioned previously), which did not have entry, then again the request will go to the backend. Now there can be a scenario wherein the first few requests forwarded by primary GFE were to a different secondary GFEs everytime, which did not have data pertaining to that URL in their cache and all the requests had to be forwarded to the backend.
Also referring to your concern about hitting cache from same browser, this behaviour is intended because Google Cloud uses anycast virtual IPs to load balance CDN traffic (which also explains the behaviour mentioned in the example above). Some of the other CDN providers load balance at the DNS level so all requests goes to the same edge server.
Is it possible to create a single policy that will forward all Options requests to the backend server? The only way I have figured out how to do this is by creating a new Options operation for each endpoint. That seems like a lot of unnecessary work since I want all options requests to be forwarded to the backend.
How can I create a single policy to forward all Options requests to the backend?
I don't think that is currently possible, so your only options are:
Create a separate OPTIONS operation in each endpoint.
Set up CORS (https://msdn.microsoft.com/library/azure/7689d277-8abe-472a-a78c-e6d4bd43455d#CORS) policy on a product or global level, so that OPTIONS requests would be served by APIM and not forwarded to backend at all.
Actually it is, operation template support wildcard /*
I am trying this, but since our API requires an SubscriptionKey, it is still not working. Since ApiMgmt will return 401 for options request, since browsers dont send the SubscriptionKey header.
Would be nice with the possibility to open single operations...
https://feedback.azure.com/forums/248703-api-management/suggestions/19450117-feature-to-ignore-api-management-subscription-key
In DevTools on the Timing tab you can see the following states:
_
Except 'Queueing' all states are explained in the DevTools documentation. Do you know what browser does in the Queueing phase ? What queue are we talking about ?
Thx
Additionally it seems that Queueing can take place at the beginning of connection setup and also at the beginning of the 'Request/Response' phase ?
From: Chrome Developers - Network features reference
Queuing
If a request is queued it indicated that:
The request was postponed by the rendering engine because it's considered lower priority than critical resources (such as scripts/styles). This often happens with images.
The request was put on hold to wait for an unavailable TCP socket that's about to free up.
The request was put on hold because the browser only allows six TCP connections per origin on HTTP 1.
Time spent making disk cache entries (typically very quick.)
Update from #cyptus comment:
With chrome v76 the network tabs will hide the CORS preflight (OPTIONS) request. The request that triggered this CORS will include the time the (invisible) OPTIONS request took in the queueing timing.
Update from #tamilsweet comment:
To show the CORS preflight follow Chrome not showing OPTIONS requests in Network tab
Another possibility may be there is a fresh service worker which is trying to install or activate. Keep the service worker url solid among your page visits.
I have two websites http://www.example.com and https://www.example.com. I am using HTML5 session storage to store user preferences.
A user arrives at http://www.example.com and I load some default settings via ajax.
They browse to a page requiring login and are sent to https://www.example.com/login.html
After they are done logging in they are sent back to http://www.example.com where because they are now logged in I should fetch new settings from the server. The trouble is that http and https are different origins and can't share session storage.
Things I've tried that don't work:
Loading a page http://www.example.com/clearSession.html in an iframe that just runs sessionStorage.removeItem('key') to clear my data, but it seems that this has it's own browsing context so it doesn't work.
Things I've tried that work but I'm not wanting to use:
Using a cookie. This works great because http and https can share cookies but this means all my user settings get sent to the server with every resource request. This is usually about 4k but could be up to 1MB of data. No I can't host my resources on a different domain.
Don't cache the settings and just make the request every time to get the settings. I am doing this on older browsers as they don't support session storage but it slows down the page load and puts extra load on my database.
I can tell you how we have solved this problem, but it doesn't involve local sessionStorage. We use a server-side session to store the user's login data (username, ID, etc.) after they have been to our authentication server and back. Before they are authenticated you could still collect preference data from them by using AJAX to report these preferences back to a web service on the server that can store it in the server's session scope. This would break the RESTful model, however, because it would assume the use of server side sessions. That would depend on your server language and how you have your web services set up.
I think you will always bump into that origin problem because that is a restriction designed into local storage in general.
Switch everything to https, its a standard now.
If your API and Website making ajax calls to that API are on the same server (even domain), how would you secure that API?
I only want requests from the same server to be allowed! No remote requests from any other domain, I already have SSL installed does this mean I am safe?
I think you have some confusion that I want to help you clear up.
By the very fact that you are talking about "making Ajax calls" you are talking about your application making remote requests to your server. Even if your website is served from the same domain you are making a remote request.
I only want requests from the same server to be allowed!
Therein lies the problem. You are not talking about making a request from server-to-server. You are talking about making a request from client-to-server (Ajax), so you cannot use IP restrictions (unless you know the IP address of every client that will access your site).
Restricting Ajax requests does not need to be any different than restricting other requests. How do you keep unauthorized users from accessing "normal" web pages? Typically you would have the user authenticate, create a user session on the server, pass a session cookie back tot he client that is then submitted on every request, right? All that stuff works for Ajax requests too.
If your API is exposed on the internet there is nothing you can do to stop others from trying to make requests against it (again, unless you know all of the IPs of allowed clients). So you have to have server-side control in place to authorize remote calls from your allowed clients.
Oh, and having TLS in place is a step in the right direction. I am always amazed by the number of developers that think they can do without TLS. But TLS alone is not enough.
Look at request_referer in your HTTP headers. That tell you where the request came from.
It depends what you want to secure it from.
Third parties getting their visitors to request data from your API using the credentials those visitors have on your site
Browsers will protect you automatically unless you take steps to disable that protection.
Third parties getting their visitors to request changes to your site using your API and the visitors' credentials
Nothing Ajax specific about this. Implement the usual defences against CSRF.
Third parties requesting data using their own client
Again, nothing Ajax specific about this. You can't prevent the requests being made. You need authentication/authorisation (e.g. password protection).
I already have SSL installed does this mean I am safe
No. That protects data from being intercepted enroute. It doesn't prevent other people requesting the data, or accessing it from the end points.
you can check ip address, if You want accept request only from same server, place .htaccess in api directory or in virtualhost configuration directive, to allow only 127.0.0.1 or localhost. Configuration is based on what webserver You have.