Related
I am currently working on a function to assist in preparing Link: HTTP header or a set of <link> tags and while reading different materials on this, I still am not able to find an answer to simple question: when to use Link: header and when to use <link>.
So far I can only say, that if you want to use HTTP20 server push, it is recommended to utilize the header. On the other hand, even if I push a stylesheet, it will not be applied unless there is a respective tag in HTML output.
Since I am preparing the library in order to help with some standardization and sanitization, I would like to catch, at least, some "weird" cases like this, if it's possible, but for that I need some set of recommendations or best practices in that regard. Sadly I am unable to find any thus far, so am turning to more knowledgeable people: what best practices or weird cases should I consider catching or should I just allow whatever to be sent regardless of whether it's a header or a tag?
If anyone is interested, the code is present in https://github.com/Simbiat/HTTP20/blob/main/src/Headers.php (links function).
They are supposed to be equivalent as #Evert states so in theory you can use either. However there are some considerations:
Headers are usually set in web server config (at least for static pages) which may not be as easy to update for developers.
However it has the added advantage that you can set these for multiple pages all at once (e.g. preload your core fonts on every .html file, rather than having to remember to set this on all pages, or all page templates if using a CMS).
On the other side with the HTML version it’s often easier to configure it per page (or page template), if you have different needs (e.g. different fonts are used in different pages).
There’s also some which say there are slight performance considerations to doing it in the header but honestly, as long as it’s high enough in the <HEAD> element I really think you’d struggle to notice this.
Of perhaps of more importance is whether it’s passed on hop to hop if your web server is hidden behind other infrastructure (e.g. a CDN or other proxy). In theory it should be, for simple headers, but for things like HTTP/2 push that’s not so easy. If it’s in the HTML you don’t need to worry about this (assuming intermediaries are not changing the markup of course!).
You mentioned the HTTP/2 push use case and that definitely needs the header (though this is not a defined standard method of setting push and some servers or CDNs use other methods, but many use this). However given HTTP/2 push’s complexities and concerns it can cause more problems than it solves, this is maybe a reason to recommend the HTML method to ensure it’s never pushed.
All in all I recommend setting this in the HTML. It’s just easier.
This is not the case however with other, similar things, which can be set in HTML and HTTP headers. CSP for example is limited in the HTML version, lacking some features of the HTTP Header version, and is also not recommended as it could be altered with JavaScript whereas the HTTP header cannot. But for simple Link headers these are less of a concern.
Were working in a component(module)/template framework.
There is only one template per page, this defines the basic structure and layout. the HEAD area is defined here.
Now, many of our components(modules) include some concept of pagination.
Thus, it's desired to use rel-next and rel-prev in the head of the document.
The problem comes from the template is (and cannot) be aware of the component that provides pagination. They are 100% completely decoupled.
Once the component is run, the head part of the page is typically flushed already.
It's just a limitation of the framework.
since placing the links in the BODY (where the component(module) renders) will not achieve the correct results (i.e. Google ignores it unless in the head).
Can anyone think of an approach or work-around to this issue?
You could send HTTP Link headers instead:
Link: <http://www.example.com/favorite-books/everything-on-one-page>; rel="canonical"
Link: <http://www.example.com/favorite-books/page-1>; rel="prev"
Link: <http://www.example.com/favorite-books/page-3>; rel="next"
According to Google’s documentation about their usage of canonical, it’s supported.
While support isn’t mentioned on Google’s documentation about their usage of prev/next, support was confirmed in a thread on their product forums.
According to Google the default option is to nothing - "Leave whatever you have exactly as-is. Paginated content exists throughout the web and we’ll continue to strive to give searchers the best result, regardless of the page’s rel=”next”/rel=”prev” HTML markup—or lack thereof."
Other more up to date mentions of the issue suggest the advice hasn't change:
Video about pagination
Infinite scroll
I am confused on how a REST api can both be hypertext driven, but also machine readable. Say I design an API and some endpoint lists contents of a collection.
GET /api/companies/
The server should return a list with company resources e.g:
/api/companies/adobe
/api/companies/microsoft
/api/companies/apple
One way would be to generate a hypertext (html) page with <a> links to the respective resources. However I would also like to make it easy for a non-browser client to read this list. For example, some client might want to populates a dropdown gui with companies. In this case returning a html document is inappropriate, and it might be better to return a list in JSON or XML format.
It is not clear to me how REST style can satisfy both. Is there a practical solution or examples of a REST api that is both nice to browsers and non-browser clients?
What you're looking for is nowadays referred to as HATEOAS API's. See this question for examples: Actual examples for HATEOAS (REST-architecture)
The ReST architectural style, as originally defined by Roy Fielding, prescribes "hypertext as the engine for application state" as one of the architectural contraints. However, this concept got more or less "lost in translation" when people started equaling "RESTful API's" with "using the HTTP verbs right" (plus a little more, if you're lucky). (Edit: Providing credence for my assertion are the first and second highest-ratest answers in What exactly is RESTful programming? . The first talks only about HTTP verbs).
Some thoughts on your question: (mainly because the subject keeps fascinating me)
In HATEOAS, standardized media types with precise meaning are very important. It's generally thought to be best to reuse an existing media type when possible, to benefit from general understanding and tooling around this. One popular method is using XML, because it offers both general structure for data and a way to define semantics, i.e. through an XML schema or with namespaces. XML in and by itself is more or less meaningless when considering HATEOAS. The same applies for JSON.
For supporting links, you want to choose a media type that either supports links "natively" (i.e. text/html, application/xhtml+xml) or a media type that allows defining what pieces in the document must be interpreted as links through some embedded metadata, such as XML can with for example XLINK. I don't think you could use application/json because JSON by itself has no pre-defined place to define metadata. I do think that it would be possible to design a media type based on json - call it application/x-descriptive-json - that defines up-front that the JSON document returned must consist of a "header" and "body" property where header may contain further specified metadata. You could also design a media type for JSON just to support embedded links. Simpler media type, less extenisble. I wouldn't be surprised if both things I describe already exist in some form.
To be both nice to browsers and non-browser clients, all it takes is respecting the Accept header. You must assume that a client who asks for text/html is truly happy with text/html. This could be an argument for not using text/html as the media type for your non-browser API entry point. In principle, I think it could work though if the only thing you want is links. Good HTML markup can be very well consumed by non-browser clients. HTML also defines way to do paging, through rel="next", rel="previous".
The three biggest problems of a singular media type for both browsers and non-browsers I see are:
you must ensure all your site html is outputted with non-browser consumption in mind, i.e. embed sufficient metadata. Perhaps add hidden links in some places. It's a bit comparable from thinking about accessibility for visually impaired people: Though now, you're designing for a consumer who cannot read English, or any natural language for that matter. :)
there may be lots of markup and content that may essentially irrelevant to a non-browser client. Think of repeating header and footer text, navigation area's that kind of things.
HTML may simply lack the expressiveness you need. In principle, as soon as you go "think up" some conventions specific to your site (i.e. say rel="original-image means the link to the full-size, original image), then you're not doing strictly HATEOS anymore (at least, that's my understanding). HTML leaves no room for defining new meaning to elements. XML does.
A work-around to problem three might be using XHTML, since XHTML, by the virtue of being XML, does allow specifying new kinds of elements through namespaces.
I see #robert_b_clarke mentioning Microformats, which is relevant in this discussion. It's indeed one way of trying to improve accessibility for non-human agents. The main issue with this from a technical point of view is that it essentially relies on "out-of-band" information. Microformats are not part of the text/html spec. In a way, it's comparable to saying: "Hey, if I say that there's a resource with type A and id X, you can access it at mysite.com/A/X." The example I gave with rel=original-image could be called a micro-format as well. But it is a way to go. "State in your API docs: We serve nicely formatted text/html. Our text/html also embeds the following microformats: ..." You can even define your own ones.
I think the following presentation as a nice down-to-earth explanation of HATEOAS:
http://www.slideshare.net/apigee/hateoas-101-opinionated-introduction-to-a-rest-api-style
Edit:
I only now read about HTML5 microdata (because of #robert_b_clarke). It seems like HTML5 does provide a way for supplying additional information beyond what's possible with standard HTML tags. Consider what I wrote dated. :) Edit edit: It's only a draft, phew. ;)
Edit 2
Re a "descriptive JSON" format: This has just been announced http://jsonapi.org/ . They have applied for their own mime type. It's by Yehuda Katz (Ember.js) and Steve Klabnib, who's writing Designing Hypermedia API's.
The HTTP Accept header can be used by clients to request a response in a specific content type. For example, your REST API clients might request JSON data using the following header:
GET http://yourdomain.com/api/companies
Accept: application/json
So your server app can then serve JSON or HTML for the same URL depending on the value of the Accept header. Of course all your REST client apps will have to include that header, which may or may not be practical.
There are numerous alternative approaches, one of which is to serve the same XHTML content to both browsers and client apps. You can use HTML5 microdata or Microformats to embed structured data within HTML. That approach has a number of limitations. API client requests will result in larger, more complicated responses than necessary as they will include a load of stuff that's only usable by a web browser. There are also other differences in behaviour you might like to enforce. For instance you would probably want an unauthorized GET request for a protected resource to result in an HTTP 401 response for a machine client, and a redirect to login page for a web browser.
You may find that the easiest way is to be less principled and serve the human friendly and machine friendly versions of your resources through separate URLs
http://yourdomain.com/companies
http://yourdomain.com/api/companies
I've seen this question answered several ways. Some developers add a request parameter to indicate the format of the response, as in /api/companies/?rtnType=json. This method may be acceptable in a small application. It is a departure from true RESTful theology though.
The better way (in Java at least) is to use something like the Spring Framework. Spring can provide dynamic response formatting based on the media type in the HTTP request. The book "Spring in Action" (Walls, 2011) has an excellent explanation of this in chapter 11. And there are similar ways to accomplish dynamic response formatting in other languages without breaking REST.
Why are iframes considered dangerous and a security risk? Can someone describe an example of a case where it can be used maliciously?
The IFRAME element may be a security risk if your site is embedded inside an IFRAME on hostile site. Google "clickjacking" for more details. Note that it does not matter if you use <iframe> or not. The only real protection from this attack is to add HTTP header X-Frame-Options: DENY and hope that the browser knows its job.
If anybody claims that using an <iframe> element on your site is dangerous and causes a security risk, they do not understand what <iframe> element does, or they are speaking about possibility of <iframe> related vulnerabilities in browsers. Security of <iframe src="..."> tag is equal to <img src="..." or <a href="..."> as long there are no vulnerabilities in the browser. And if there's a suitable vulnerability, it might be possible to trigger it even without using <iframe>, <img> or <a> element, so it's not worth considering for this issue.
In addition, IFRAME element may be a security risk if any page on your site contains an XSS vulnerability which can be exploited. In that case the attacker can expand the XSS attack to any page within the same domain that can be persuaded to load within an <iframe> on the page with XSS vulnerability. This is because vulnerable content from the same origin (same domain) inside <iframe> is allowed to access the parent content DOM (practically execute JavaScript in the "host" document). The only real protection methods from this attack is to add HTTP header X-Frame-Options: DENY and/or always correctly encode all user submitted data (that is, never have an XSS vulnerability on your site - easier said than done).
However, be warned that content from <iframe> can initiate top level navigation by default. That is, content within the <iframe> is allowed to automatically open a link over current page location (the new location will be visible in the address bar). The only way to avoid that is to add sandbox attribute without value allow-top-navigation. For example, <iframe sandbox="allow-forms allow-scripts" ...>. Unfortunately, sandbox also disables all plugins, always. For example, historically Youtube couldn't be sandboxed because Flash player was still required to view all Youtube content. No browser supports using plugins and disallowing top level navigation at the same time. However, unless you have some very special reasons, you cannot trust any plugins to work at all for majority of your users in 2021, so you can just use sandbox always and guard your site against forced redirects from user generated content, too. Note that this will break poorly implemented content that tries to modify document.top.location. The content in sandboxed <iframe> can still open links in new tabs so well implemented content will work just fine. Also notice that if you use <iframe sandbox="... allow-scripts allow-same-origin ..." src="blog:..."> any XSS attack within the blob: content can be extended to host document because blob: URLs always inherit the origin of their parent document. You cannot wrap unfiltered user content in blob: and render it as an <iframe> any more than you can put that content directly on your own page.
Example attack goes like this: assume that users can insert user generated content with an iframe; an <iframe> without an attribute sandbox can be used to run JS code saying document.top.location.href = ... and force a redirect to another page. If that redirect goes to a well executed phishing site and your users do not pay attention to address bar, the attacker has a good change to get your users to leak their credentials. They cannot fake the address bar but they can force the redirect and control all content that users can see after that. Leaving allow-top-navigation out of sandbox attribute value avoids this problem. However, due historical reasons, <iframe> elements do not have this limitation by default, so you'll be more vulnerable to phishing if your users can add <iframe> element without attribute sandbox.
Note that X-Frame-Options: DENY also protects from rendering performance side-channel attack that can read content cross-origin (also known as "Pixel perfect Timing Attacks").
That's the technical side of the issue. In addition, there's the issue of user interface. If you teach your users to trust that URL bar is supposed to not change when they click links (e.g. your site uses a big iframe with all the actual content), then the users will not notice anything in the future either in case of actual security vulnerability. For example, you could have an XSS vulnerability within your site that allows the attacker to load content from hostile source within your iframe. Nobody could tell the difference because the URL bar still looks identical to previous behavior (never changes) and the content "looks" valid even though it's from hostile domain requesting user credentials.
As soon as you're displaying content from another domain, you're basically trusting that domain not to serve-up malware.
There's nothing wrong with iframes per se. If you control the content of the iframe, they're perfectly safe.
I'm assuming cross-domain iFrame since presumably the risk would be lower if you controlled it yourself.
Clickjacking is a problem if your site is included as an iframe
A compromised iFrame could display malicious content (imagine the iFrame displaying a login box instead of an ad)
An included iframe can make certain JS calls like alert and prompt which could annoy your user
An included iframe can redirect via location.href (yikes, imagine a 3p frame redirecting the customer from bankofamerica.com to bankofamerica.fake.com)
Malware inside the 3p frame (java/flash/activeX) could infect your user
IFRAMEs are okay; urban legends are not.
When you "use iframes", it doesn't just mean one thing. It's a lexical ambiguity. Depending on the use case, "using iframes" may mean one of the following situations:
Someone else displays your content in an iframe
You display domeone else's content in an iframe
You display your own content in an iframe
So which of these cases can put you in risk?
1. Someone else displays your content
This case is almost always referred to as clickjacking - mimicking your site's behaviour, trying to lure your users into using a fake UI instead of the real site. The misunderstanding here is that you using or not using iframes is irrelevant, it's simply not your call - it's someone else using iframes, which you can do nothing about. Btw, even they don't need them specifically: they can copy your site any other way, stealing your html, implementing a fake site from scratch, etc.
So, ditching iframes in attempt to prevent clickjacking - it makes exactly zero sense.
2. You display someone else's content
Of the three above, this is the only one that's somewhat risky, but most of the scary articles you read all the time come from a world before same-origin policy was introduced. Right now, it's still not recommended to include just any site into your own (who knows what it will contain tomorrow?), but if it's a trusted source (accuweather, yahoo stock info etc), you can safely do it. The big no-no here is letting users (therefore, malicious users) control the src of the iframe, telling it what to display. Don't let users load arbitrary content into your page, that's the root of all evil. But it's true with or without iframes. It has nothing to do with them; it could happen using a script or a style tag (good luck living without them) - the problem is you let them out. Any output on your site containing any user-given content is RISKY. Without sanitizing (de-HTMLifying) it, you're basically opening your site up for XSS attacks, anyone can insert a <script> tag into your content, and that is bad news. Like, baaaad news.
Never output any user input without making dead sure it's harmless.
So, while iframes are innocent again, the takeaway is: don't make them display 3rd-party content unless you trust the source. In other words, don't include untrusted content in your site. (Also, don't jump in front of fast-approaching freight trains. Duuh.)
3. You display your own content in an iframe
This one is obviously harmless. Your page is trusted, the inner content of the iframe is trusted, nothing can go wrong. Iframe is no magic trick; it's just an encapsulation technique, you absolutely have the right to show a piece of your content in a sandbox. It's much like putting it inside a div or anything else, only it will have its own document environment.
TL;DR
Case 1: doesn't matter if you use iframes or not,
Case 2: not an iframe problem,
Case 3: absolutely harmless case.
Please stop believing urban legends. The truth is, iframe-s are totally safe. You could as well blame script tags for being dangerous; anything can cause trouble when maliciously inserted in a site. But how did they insert it in the first place? There must be an existing backend vulnerability if someone was able to inject html content into a site. Blaming one piece of technology for a common attack (instead of finding the real cause) is just a synonym for keeping security holes open. Find the dragon behind the fire.
Unsanitized output is bad; iframes are not.
Stop the witch-hunt.
UPDATE:
There is an attribute called sandbox, worth checking out: https://www.w3schools.com/tags/att_sandbox.asp
UPDATE 2:
Before you comment against iframes - please think about hammers. Hammers are dangerous. They also don't look very nice, they're difficult to swim with, bad for teeth, and some guy in a movie once misused a hammer causing serious injuries. Also, just googled it and tons of literature says mortals can't even move them. If this looks like a good reason to never ever use a hammer again, iframes may not be your real enemy. Sorry for going offroad.
"Dangerous" and "Security risk" are not the first things that spring to mind when people mention iframes … but they can be used in clickjacking attacks.
iframe is also vulnerable to Cross Frame Scripting:
https://www.owasp.org/index.php/Cross_Frame_Scripting
My web-app has to deliver db-generated content by email. The HTML need not include javascript but _does have to include images and CSS.
It's a safe bet that most users will be viewing this in Outlook. Are general guidelines different for html destined for email vs. IE?
Outlook doesn't have a quite good reputation about HTML mails -- see Microsoft to ignore web standards in Outlook 2010 - enough is enough, for instance.
But it's not the only email client with problems, actually : generating an HTML e-mail generally means doing some things differently than for a webpage...
For instance, you can take a look at this article : Ensuring your HTML Emails Look Great and Get Delivered. A few points it make are :
use inline CSS
use tables and basic layout
Well... quite the contrary of what is said for webpages, isn't it ?
And a piece of advice : always test your email in as many clients (both desktop-based, like outlook, thunderbird, ... and web-based, like yahoo, hotmail, gmail, ... ) as possible !
Absolutely. It's virtually impossible to use CSS layout techniques in HTML emails. So...code like it's 1998.
Use tables. It's painful, but it's the only surefire way to get HTML emails to display properly and consistently. Make sure you set cellpadding, cellspacing, and border on every single table. Nest them if you must. colspan and rowspan are your friends. Use shim .gifs in empty cells. If it's a bad practice for regular webpages, you should probably be using them in e-mails.
Don't design for anything wider than 600 pixels.
Since webmail services' CSS can be overwritten by CSS inside of the e-mails, they tend to disable it. Inline CSS is smart. Font tags are smarter.
Forget about doctypes, <html>, <body>, or <head>.
Don't rely on background images - if you must use them, have a background color backup.
Avoid spammy-sounding terms. Don't talk about Viagra, porn, or knockoff watches - it'll probably get spam-filtered.
Send text-only alternative using the "Multipart/Alternative" MIMEtype.
Debugging is a bear. The testing list we use at my place of business: Outlook 2003, Outlook 2007, Outlook Express (from XP), Thunderbird Mac and Windows, Eudora, Mail.app, Entourage, Gmail, Yahoo! Mail, AOL, AOL Webmail, and Hotmail. Webmail clients tested in both IE6 and the latest Firefox version. It may be worth cooking up a script to automatically send emails to test addresses at all the services for testing.
Good info about specific support at Campaign Monitor's site. They also have more practical guidelines. And here's a list of e-mail client usage.
In Outlook 2007, it took a huge step backward: http://www.sitepoint.com/blogs/2007/01/10/microsoft-breaks-html-email-rendering-in-outlook/
Instead of using IE for the rendering engine, it now uses Word. Only the most basic HTML is supported well or at all.
http://www.campaignmonitor.com/blog/post/2533/a-guide-to-css-support-in-emai-2/
Email user agents, in general, tend to diverge from the standards for CSS and HTML pretty significantly for good reason. Some email user agents, such as Thunderbird, claim to have full CSS support in email, but to my knowledge no email user agent fully supports the HTML standard.
HTML is designed for the web, specifically, HTTP transmission. It contains header definitions, all the bits that fall under the <head></head> tags, that are intended to operate under HTTP transmission. HTML is a markup language, which is a document of structured data. Markup languages are not presentation languages, but the only point of using HTML in email is for presentation. As a result HTML is not functional in email and is hardly accessible.
In HTTP a user requests a resource from a server and a server responds with that resource. The author of the document provided from the server is somebody who is able to author documents on/to the server. A resource provided by HTTP has only one author.
In email a document can be written to by many different users opposed to HTTP. A document does not originate at a server. Since a document does not originate at a server or a known single user markup languages and CSS experience containment and scope problems. This means HTML and CSS in email are entirely incompatible to the concept of an email thread where multiple people are contributing to a document, because the CSS does not understand that it is supposed to be constrained to a single instance of communication.
Due to these incompatibilities that were never intended to exist and are not intended to ever be corrected Microsoft is entirely correct in supporting as little or as much HTML in email as they want. There is no standard addressing this, so there is no violation. The violation is using HTML in email to begin with.
My advise is to wait until Mail Markup Language is adopted. If we are all very lucky one particular user agent vendor may announce adoption of Mail Markup Language in the near future. To my knowledge Mail Markup Language is the only functional markup language specification to address the requirements of email threads at this time.