When I request a webpage, I actually make lots of requests for imgs, banners, javascripts (js can include others js and so on), iframes, css, plugins, activex, applets,...
How can I get the url's list I'm actually downloading from?
Easiest way is to use your browser's built-in development tools. Generally look for the "network" tab.
(Google Chrome's Network Panel)
You can also use a tool such as Fiddler, which acts as a proxy server and shows you all web requests, even HTTP from other applications.
Related
I have the problem, that my website acts different if I call it via the https protocol. To be more precise it looks like it handles the CSS in a different way.
What I want (and how it actually works via http) is kind of a navigation with different tabs. Here is an image of the navigation part:
http navigation
And here is an image of how it looks like when it's called via https:
https navigation
I have also created a fiddle with this part of my website although it does not proper work inside the fiddle. (maybe because the jsfiddle site is also via https protocol?)
Fiddle
However, please have a look at the current website to see the difference:
This is the website via http:
http website
and here how it looks like when it's called via https:
https website
You guys ever had a similar problem or any idea how to solve it?
I was the opinion that the protocol should not make a difference.
Take a look at your console.
You should see a lot of Mixed Content errors or warnings if you are using Chrome.
When a website is served over HTTPS, all its resources must be served over HTTPS too. When a resource is not loaded over HTTPS, the browser will block it because otherwise it defeats the whole purpose of using a HTTPS.
When one resource is blocked, the content from these resources won't get executed. Maybe that is the reason why your layout breaks because something is not being executed properly.
So try to change your resources into HTTPS protocol. If you are using APIs and those API does not provide HTTPS link, then you should look for another API.
In your case
This is the culprit.
http://fonts.googleapis.com/css?family=Source+Sans+Pro:300,400,700,300italic,400italic,700italic
You can find it in head section.
Your font should be in Source Sans Pro but because it was blocked, Helvetica or Arial was used instead. Thus breaking the layout.
Change it to HTTPS and it should be fine.
I don't know much about HTTP requests, but I can tell you two things:
- The browser cascades and parses CSS in different ways. HTTPS requests are processed in different ways. Maybe check you cascade.
- That website looks nice
Is is possible, on the client side, to manipulate the HTML sent from a site when you are not the owner of the site, and have no access to the code?
Yes. There are many ways.
HTTP proxies such as Fiddler or Burp can break on all requests and then manipulate the response before it reaches the browser.
Video explaining how this is done in Fiddler.
There are also several browser plugins like Tampermonkey (Chrome) and Greasemonkey (Firefox) that allow you to execute custom JavaScript on page load automatically.
There are built-in DOM manipulation extensions for all browsers, like this one :
https://addons.mozilla.org/en-US/firefox/addon/greasemonkey/
While this could be a very basic thing, I can't seem to find an answer to this one. I see a lot of discussion about content scripts. When I open web inspector in chrome, and select sources, I see a tab called content scripts. I see a couple of random numbers and a number of scripts and I can't seem to figure out what these are.
How are these being shown? where are these coming from? I can't see my server serving any of these.
In Google Chrome, content scripts are JavaScript files that are part of browser extensions. They operate on the web page much like regular javascript does, but from a protected scope (Google calls it an "Isolated World").
Content scripts have a few more privileges than ordinary javascript and, for that reason, content-script JS and page JS cannot ordinarily interact with each other.
When looking at that Content Scripts tab, you will see both the scripts added by any extensions (or by userscripts which are compiled into extensions in Chrome), and also content scripts that are built-in parts of the browser, specifically the API that extensions can use.
You can match extensions to those "random numbers" by opening the extensions tab (chrome://chrome/extensions), and activating Developer mode:
Early on I was doing some debugging and testing using the chrome dev tools(known as inspect element). I found out that on the Resources column of the dev tool, Chrome can always access the resources from the server and display them(links, videos, images....). Just wonder how Chrome does that. Is there any way to write a piece of code doing the same thing(access the server resources of other websites, not modifying them but displaying, stuff like, the link of the video currently playing on the website, which usually does not pop up until the play button is hit)?
DevTools doesn't fetch resources from a site. It fetches them from the browser.
There were similar questions already
How does webkit/chrome's WebInspector Resources Inspection work?
and
Getting Information from Google Chrome's Developer Tools
The Chrome Developer Tools has two parts frontend (html+javascript) and backend (C++) and there is an API which specifies the protocol between these parts. You can see the specification here. You can use it for writing your own app instead of standard DevTools frontend.
Also there is experimental debugging API for chrome extensions.
I think the Webkit WebInspector go over the hole source code and match all resources of the source.
So it match <link href="something.css"> and then it place something.css in the resource panel under stylesheets. And exactly the same thing for the other tags.
It's not hard to make regexes for this.
In Firefox, I view my site and get no warnings about insecure mixed content.
Using FireBug, I can see that every request is https.
In Chrome, I get the https crossed out in the address bar.
I viewed source in Chrome and then ran this regex /http(?!s)/ but the only things it found were the href attributes for some external links and the doc type and http-equiv meta tags.
Using Chrome's Resource Tracking revealed all requests were https too.
This includes Google Analytics, jQuery from Google's CDN and Facebook like scripts.
Is there any specific tool I can use to show non https requests, or anything further I can try?
I found that I get the "mixed content"-warning in Chrome even when there is no mixed content, if sometime during the session mixed content was already encountered on the domain.
(Also mentioned here: Why is Chrome reporting a secure / non secure warning when no other browsers aren't?)
In Chrome's Developer Tools, the Console tab shows the resources that it won't load because they unsecure.
You can add the "scheme" column to the Chrome developer tools network tab to show which requests were sent over http or https:
Press F12 to show the developer tools
Switch to the Network tab
Right click in the column headers and select "Scheme"
Reload the page to show which elements are loaded over http or https
In situations like this where it's helpful to see exactly which protocol is being used to load resources, I would recommend Fiddler2 as a browser-agnostic solution that can show you exactly what traffic is occurring on each request.
From the site:
Fiddler is a Web Debugging Proxy which logs all HTTP(S) traffic between your computer and the Internet. Fiddler allows you to inspect all HTTP(S) traffic, set breakpoints, and "fiddle" with incoming or outgoing data. Fiddler includes a powerful event-based scripting subsystem, and can be extended using any .NET language.
Edit: In-browser debugging tools are becoming really good so this third-party tool may not be as useful as it was when this answer was first written.
Open up the Web Inspector and find the yellow triangle (warning) in the top right. Click on it and it will display all security issues.
In 48-th version of chrome they added a security panel. Using it you can quickly identify the mixed content resources:
Do you have the HttpFox plugin for FireFox? That'd work, I think.
Among other things, it reports on the URL, Method, Result Code, and bytes of all the assets that a web page requests. It's what I've used to trap the occasional non-HTTPS graphic, etc. I'm sure the other suggested tools would do the same...
You can use SslCheck
It's a free online tool that crawls a website recursively (following all internal links) and scans for nonsecure includes - images, scripts and CSS.
(disclaimer: I'm one of the developers)
I know this post is old, but I ran across it and had the same issue. I clicked on the Chrome menu (top right corner), scrolled down to Tools> and selected Developer Tools. Clicked on the Console tab and it told me exactly what the problem was... the favicon was served over http, not https, but of course it was not in the page source code. Corrected the problem in my CMS, which loads the favicon without code in the page... and no more error!
Note that 'mixed content' and 'mixed scripting' are detected seperatly. Check this site for the meaning of the icons in Chrome: https://support.google.com/chromebook/answer/95617?p=ui_security_indicator&rd=1 (click 'see details' link).
Grey icon = mixed content, red icon = mixed scripting.