I have a system whenever user upload an image, it will send an email to the registered user's gmail. But in the email, i see something like this, the thumbnail is not viewable.
I inspect on the element, and found the src linked to this url:
https://ci5.googleusercontent.com/proxy/VI2cPXWhfKZEIarh-iyKNz1j9q7Ymh8ty4Yz19lXh82RjSlACBzS0aRajfIj913uXAsX2ylcLEDs5FBsj4cR9TcU75Pw5djdHx4htxdCAQxs_ue1Q1wi5TV43uLLBpigpjH1xN747mUHSRdTBJmXQWFyykInJCRXicM1KhNk=s0-d-e1-ft#https://www.somedomain.com/files/1658/thumbnail_71JtDozxS1L._SY450_.jpg
Obviously it is being cached by google proxy
But i can view the image without google user content, by accessing https://www.somedomain.com/files/1658/thumbnail_71JtDozxS1L._SY450_.jpg (i masked the domain so the image might not available to you).
I tried to clear browser cache but the problem still persist. How can i bypass the googleusercontent thingy or at least make the thumbnail able to display.
I checkout on this link Images not displayed for Gmail but im not using localhost and the image itself is accessible outside of my local network.
How does Google Image Proxy work
The Google Image Proxy is a caching proxy server. Every time an image link is included in email the request will go to the Google Image Proxy first to see if it has been cached, if so it should serve it up from the proxy or it will go fetch it and cache it there after.
The solution for most issues
The Google Image Proxy server will fetch your images if this images:
have extensions like .png, .jpg/.jpeg or .gif only. May be .webp too. But not .svg.
do not use any kind of query string part in the image URL like ?id=123
have an URL which is mapped onto the image directly.
have not a long name.
Requirements for image server:
The response from image server/proxy server must include the correct header like Content-Type: image/jpeg.
File extension and content-type header must be in the same type.
Status code in server response must be 200 instead of 403, 500 and etc.
What could help too?
Google support answer:
Set up an image URL proxy whitelist
When your users open email messages, Gmail uses Google’s secure proxy
servers to serve images that might be included in these messages. This
protects your users and domain against image-based security
vulnerabilities.
Because of the image proxy, links to images that are dependent on
internal IPs and sometimes cookies are broken. The Image URL proxy
whitelist setting lets you avoid broken links to images by creating
and maintaining a whitelist of internal URLs that'll bypass proxy
protection.
When you configure the Image URL proxy whitelist, you can specify a
set of domains and a path prefix that can be used to specify large
groups of URLs. See the guidelines below for examples.
Configure the Image URL proxy whitelist setting:
Sign in to your Google Admin console. Sign in using your administrator account (does not end in #gmail.com).
From the Admin console Home page, go to Apps > G Suite > Gmail > Advanced settings. Tip: To see Advanced settings,
scroll to the bottom of the Gmail page.
On the left, select your top-level organization.
Scroll to the Image URL proxy whitelist section.
Enter image URL proxy whitelist patterns. Matching URLs will bypass image proxy protection. See the guidelines below for more details and
instructions.
At the bottom, click Save.
It can take up to an hour for changes to propagate to user accounts.
You can track prior changes under Admin console audit log.
Guidelines for applying the Image URL proxy whitelist setting
Security considerations
Consult with your security team before configuring the Image URL proxy
whitelist setting. The decision to bypass image proxy whitelist
protection can expose your users and domain to security risks if not
used with care.
In general, if you have a domain that needs authentication via cookie,
and if that domain is controlled by an administrator within your
organization and is completely trusted, then whitelisting that URL
should not expose your domain to image-based attacks.
Important: Disabling the image proxy is not recommended. This option is available to provide flexibility for administrators, but
disabling the image proxy can leave your users vulnerable to malicious
attacks.
Entering Image URL patterns
To maintain a whitelist of internal URLs that'll bypass proxy
protection, enter the image URL patterns in the Image URL proxy
whitelist setting. Matching URLs will bypass the image proxy.
A pattern can contain the scheme, the domain, and a path. The pattern
must always have a forward slash (/) present between the domain and
path. If the URL pattern specifies a scheme, then the scheme and the
domain must fully match. Otherwise, the domain can partially match the
URL suffix. For example, the pattern google.com matches
www.google.com, but not gle.com. The URL pattern can specify a
path that's matched against the path prefix.
Important: Enter your actual domain name as you enter the image URL pattern. Always include a trailing forward slash (/) after the
domain name.
Examples of Image URL patterns
The following patterns are examples only. The following patterns:
http://rule_fixed_scheme_domain.com/
rule_flex_scheme_domain.com/
rule_fixed_subpath.com/cgi-bin/
... will match the following URLs:
http://rule_fixed_scheme_domain.com/
http://rule_fixed_scheme_domain.com/test.jpg?foo=bar#frag
http://rule_fixed_scheme_domain.com
rule_flex_scheme_domain.com/
t.rule_flex_scheme_domain.com/test.jpg
http://t.rule_flex_scheme_domain.com/test.jpg
https://t.rule_flex_scheme_domain.com/test.jpg
http://rule_fixed_subpath.com/cgi-bin/
http://rule_fixed_subpath.com/cgi-bin/people
Note: The URL scheme (http://) is optional. If the scheme is omitted, the pattern can match any scheme, and allows partial matches
on the domain suffix.
Previewing the image URL patterns
Click Preview to see if the URLs match the image URL patterns
you've set. If the image URL matches a pattern, you'll see a
confirmation message. If the image URL does not match, an error
message appears.
Bharata has a great and detailed answer on this, but just wanted to add one addition that I identified with a similar issue.
We had a x-webkit-csp content security header that turned out to be the culprit.
Removing it and all worked through the image proxy.
Google's response was that x-webkit-csp is deprecated and to use the Content-Security-Policy header instead.
However this seems like a bug that an unsupported header throws a fatal error rather than simply ignoring it.
TL;DR: Make sure your server isn't blocking external connections (through AWS or .htaccess or some other firewall)!
I was having this problem too. I ran through every solution I could think of and every one I found online. Nothing fixed it.
Finally, I inspected the image in Gmail so that I could get the full CDN address Google creates for it. I tried to open that in a new tab and it failed. So I realized that Google wasn't able to grab the image.
In the end, I'd forgotten that I have the server locked down from all traffic except for my own (just a basic .htaccess IP deny). It's just a simple security layer I use while I'm in my development. Keep in mind that you might have it locked down within AWS or something more rugged like that.
I opened up all IPs for a minute, tested it, and sure enough it worked as expected. The old emails that were previously failing also worked, so it seems that Google tries to work their magic anytime the email is opened and they don't have the image saved. Once I closed the IP address again, the image continued to work whatever Google. I'm guessing once they write it to their CDN is remains there indefinitely.
So if you're certain that you've done everything else correctly, I would suggest making sure that the server is indeed open to the outside world so Google can process the image.
I had the same problem and I solved it by specifying the "https://" protocol in the "src" url of the img, otherwise by default "http" is prepended
Related
I have a small django website that allows file uploading hosted on pythonanywhere. The files are uploaded to aws s3.
Now the problem is that even with the presence of download attribute in the html, the browser still renders the file instead of downloading it
<a download href="{{file_url}}">Download</a>
From Mozilla Developer Network Anchor element reference:
"....download only works for same-origin URLs, or the blob: and data: schemes..."
Your aws s3 links most probably are not same-origin (same domain etc) as your site.
If that is what you are running into, one work around that comes to my mind is, you can make on your site a transfer url that receives the document identifier, downloads from aws s3, and then forwards its content as response. This way you can also control content-type like headers that you may need to explicitly set to make the browser behave the way you want.
One addition, if you decide to have a solution like that, you have to take precautions to control the request content that transfer url, and only transfer content that your web site intended to. Otherwise you will have opened a vulnerability similar to what is called an "Open Redirect" vulnerability.
I store urls in a database for the users of my webapp and I am not sure whether I need to store whether it was a "http" or a "https" request.
If I don't store the type of the connection and just echo to the users a link with "http", will it in 100% of the cases use a "https" connection automatically (when it is possible)? I don't want to be responsible for a user not using a https connection even though it is possible.
Does the browser (Chrome/Firefox) automatically use https even when you try to use http?
No. If you tell the browser to use HTTP, then it will use HTTP.
Schemes will only be added to a URL automatically under two circumstances:
When it is a relative URL, in which case the scheme will be the same as the one used to load the current document.
When the user types the URL into the browser's address bar and omits the scheme, in which case it will default to HTTP (not HTTPS).
A web server might provide HTTP and HTTPS versions of the same URL with the HTTP version containing a redirect to the HTTPS version and the HTTPS version hosting the content.
A web server might, for that matter, not provide an HTTP version at all… but that is very uncommon.
I am not sure whether I need to store whether it was a "http" or a "https" request.
You should store the full URL. You shouldn't omit bits and hope that you can fill them in by guesswork.
It won't automatically do that, but there are ways to help out:
some users may have the "HTTPSeverywhere" extension, which will attempt to redirect to HTTPS
you can serve HSTS headers, which will make the browser automatically stick to HTTPS if the user has at least once been on HTTPS with your site
Now there are a few problems with these points:
not everyone use the extension
HSTS only works once the user was visiting the URL with HTTPS and it will only work on site with HSTS headers set up, so if links are external, this might not be the case.
That being said: Are the links you store links to your own domain or external links to any web site?
My site is using HTTPS only.
I allow using BBCodes to show images. Users are placing images like "https://imagehoster.net/img.png" and the imagehoster is using a redirect so the browser loads it via HTTP "http://imagehoster.net/img.png". This makes the browser showing annoying mixed content warnings. Is there a way to prevent this?
Short: NO
Long:
the have no really web server listening to ssl.
in fact, there is only a firewall/proxy which sends a http locate to the browser.
You can't intercept that request. even if you could, where to redirect to?
they don't provide a ssl server, because it takes to much resources for encryption or it takes to much traffic, because proxy#s can't cache.
An idea to solve that problem:
detect those links, download them and store a copy on your server.
replace the link. maybe you need only to store a preview. if the click on it redirect to the original link on a new browser window.
While adding a hyperlink to another site (which has SSL), the site documentation sometimes say to link to the http:// link instead of the https:// (e.g. Play store, which is a site that uses SSL but it does not tell you to link to https; instead, it says to link to http). They do not matter (as they function normally), but would there be a reason to link to the http:// instead the https://?
Maybe they don't want extra encryption and lowering down the site speed as SSL may decrease performance somewhat.
If users are downloading large, public files, there may be a system burden to encrypt these each time.
Some browsers may not support SSL.
You will probably want the home page accessible via HTTP, so that users don't have to remember to type https to get to it.
Your specific portion of page needs secure http(https) not whole site.
Your site is indexed mainly on http on Search engines.
I've registered a domain with www.domain101.com. We'll call it www.example.com. I used their 'cloaked forwarding' option on my heroku app, which we'll call example.herokuapp.com. If I want to navigate to the posts index page, I would go to example.herokuapp.com/posts, of course; however, when I try this with my cloaked domain - www.example.com/posts - it takes me to the landing page. What's more, the url is constantly set to 'www.example.com' and the title of each page is "EXAMPLE.COM". I can click buttons on the pages and navigate through my website, but the url remains strictly at 'www.example.com'.
I used the forwarding option available on the website. What else should I be doing?
Domain cloaking uses a frame to make visitors' browsers show the cloaking domain in the location bar, while having the actual content hosted somewhere else. That's why the location bar never changes when you navigate around the site; and it isn't surprising that domain101's cloaking implementation doesn't forward path information to your Heroku app.
What you should do instead is configure a DNS server to direct traffic for your domain directly to Heroku. The simplest option for that on Heroku is the Zerigo DNS Add-on, which is free if you're hosting a single domain. You can also find DNS servers provided by lots of other people if you prefer.
Once you've configured a DNS server, you also need to tell your registrar that the DNS server you chose is the primary DNS for your domain. The way to do that varies between registrars, so you should check domain101's documentation for details.