Force no-cache on Server Side Include? - html

I have a static HTML page which includes a second HTML page, via an (Apache) include virtual.
The second page is also static, except that it includes the contents of an HTML select via an include from the server:
<select id = "idListOpts" class="form-control input-md" size = 1>
<!--#include virtual="/cgi-bin/listopts" -->
</select>
This is fine, except that the listopts script (actually compiled C++) is dynamic, and returns content from a changing database. The problem is that I can't do anything to persuade the browser (FF 27, Chrome, Opera, IE9) that the select is actually dynamic. I've tried returning various no-cache directives in the listopts header, and I'm currently using:
"Status: 200 OK\n"
"Pragma: no-cache\n"
"Cache-Control: no-cache\n"
"Cache-Control: max-age=0\n"
"Content-Type: text/html\n\n"
"<option value=\"-1\">...etc"
As far as I can make out with Firebug, the entire HTML page, including all virtual includes, is loaded only once when the page is manually refreshed, and the select contents are only fetched at that time, and included as a static list. This doesn't seem to be what the Apache docs say. The docs say that SSIs can be used when only part of a page is to be dynamically generated.
Any idea how to fix this, short of JavaScript/Ajax?

Depending on the version of Apache HTTPd being used, try using the exec element with the cgi attribute set to the script. NB, it requires mod_cgi to be enabled, but I'm assuming this is already in place.
For example:
<!--#exec cgi="/cgi/listopts" -->
If that doesn't work, I'd debug your C++ program and check that it is guaranteed to return new data on each request. If you haven't already tried this, I'd use strace or an equivalent probe application to inspect the calls being made to the listopts program and inspect the responses it returns.

Related

HTML does not display text file hosted by a specific site [duplicate]

I'm writing a tiny webpage whose purpose is to frame a few other pages, simply to consolidate them into a single browser window for ease of viewing. A few of the pages I'm trying to frame forbid being framed and throw a "Refused to display document because display forbidden by X-Frame-Options." error in Chrome. I understand that this is a security limitation (for good reason), and don't have access to change it.
Is there any alternative framing or non-framing method to display pages within a single window that won't get tripped up by the X-Frame-Options header?
I had a similar issue, where I was trying to display content from our own site in an iframe (as a lightbox-style dialog with Colorbox), and where we had an server-wide "X-Frame-Options SAMEORIGIN" header on the source server preventing it from loading on our test server.
This doesn't seem to be documented anywhere, but if you can edit the pages you're trying to iframe (eg., they're your own pages), simply sending another X-Frame-Options header with any string at all disables the SAMEORIGIN or DENY commands.
eg. for PHP, putting
<?php
header('X-Frame-Options: GOFORIT');
?>
at the top of your page will make browsers combine the two, which results in a header of
X-Frame-Options SAMEORIGIN, GOFORIT
...and allows you to load the page in an iframe. This seems to work when the initial SAMEORIGIN command was set at a server level, and you'd like to override it on a page-by-page case.
All the best!
If you are getting this error for a YouTube video, rather than using the full url use the embed url from the share options. It will look like http://www.youtube.com/embed/eCfDxZxTBW4
You may also replace watch?v= with embed/ so http://www.youtube.com/watch?v=eCfDxZxTBW4 becomes http://www.youtube.com/embed/eCfDxZxTBW4
If you are getting this error while trying to embed a Google Map in an iframe, you need to add &output=embed to the source link.
UPDATE 2019: You can bypass X-Frame-Options in an <iframe> using just client-side JavaScript and my X-Frame-Bypass Web Component. Here is a demo: Hacker News in an X-Frame-Bypass. (Tested in Chrome & Firefox.)
There is a plugin for Chrome, that drops that header entry (for personal use only):
https://chrome.google.com/webstore/detail/ignore-x-frame-headers/gleekbfjekiniecknbkamfmkohkpodhe/reviews
Adding a
target='_top'
to my link in the facebook tab fixed the issue for me...
If you're getting this error trying to embed Vimeo content, change the src of the iframe, from: https://vimeo.com/63534746 to: http://player.vimeo.com/video/63534746
I had same issue when I tried embed moodle 2 in iframe, solution is Site administration ► Security ► HTTP security and check Allow frame embedding
Solution for loading an external website into an iFrame even tough the x-frame option is set to deny on the external website.
If you want to load a other website into an iFrame and you get the Display forbidden by X-Frame-Options” error then you can actually overcome this by creating a server side proxy script.
The src attribute of the iFrame could have an url looking like this: /proxy.php?url=https://www.example.com/page&key=somekey
Then proxy.php would look something like:
if (isValidRequest()) {
echo file_get_contents($_GET['url']);
}
function isValidRequest() {
return $_SERVER['REQUEST_METHOD'] === 'GET' && isset($_GET['key']) &&
$_GET['key'] === 'somekey';
}
This by passes the block, because it is just a GET request that might as wel have been a ordinary browser page visit.
Be aware: You might want to improve the security in this script. Because hackers could start loading in webpages via your proxy script.
This is the solution guys!!
FB.Event.subscribe('edge.create', function(response) {
window.top.location.href = 'url';
});
The only thing that worked for facebook apps!
I tried nearly all suggestions. However, the only thing that really solved the issue was:
Create an .htaccess in the same folder where your PHP file lies.
Add this line to the htaccess:
Header always unset X-Frame-Options
Embedding the PHP by an iframe from another domain should work afterwards.
Additionally you could add in the beginning of your PHP file:
header('X-Frame-Options: ALLOW');
Which was, however, not necessary in my case.
It appears that X-Frame-Options Allow-From https://... is depreciated and was replaced (and gets ignored) if you use Content-Security-Policy header instead.
Here is the full reference: https://content-security-policy.com/
I had the same problem with mediawiki, this was because the server denied embedding the page into an iframe for security reasons.
I solved it writing
$wgEditPageFrameOptions = "SAMEORIGIN";
into the mediawiki php config file.
Hope it helps.
Not mentioned but can help in some instances:
var xhr = new XMLHttpRequest();
xhr.onreadystatechange = function() {
if (xhr.readyState !== 4) return;
if (xhr.status === 200) {
var doc = iframe.contentWindow.document;
doc.open();
doc.write(xhr.responseText);
doc.close();
}
}
xhr.open('GET', url, true);
xhr.send(null);
FWIW:
We had a situation where we needed to kill our iFrame when this "breaker" code showed up. So, I used the PHP function get_headers($url); to check out the remote URL before showing it in an iFrame. For better performance, I cached the results to a file so I was not making a HTTP connection each time.
I was using Tomcat 8.0.30, none of the suggestions worked for me. As we are looking to update the X-Frame-Options and set it to ALLOW, here is how I configured to allow embed iframes:
Navigate to Tomcat conf directory, edit the web.xml file
Add the below filter:
<filter>
<filter-name>httpHeaderSecurity</filter-name>
<filter-class>org.apache.catalina.filters.HttpHeaderSecurityFilter</filter-class>
<init-param>
<param-name>hstsEnabled</param-name>
<param-value>true</param-value>
</init-param>
<init-param>
<param-name>antiClickJackingEnabled</param-name>
<param-value>true</param-value>
</init-param>
<init-param>
<param-name>antiClickJackingOption</param-name>
<param-value>ALLOW-FROM</param-value>
</init-param>
<async-supported>true</async-supported>
</filter>
<filter-mapping>
<filter-name>httpHeaderSecurity</filter-name>
<url-pattern>/*</url-pattern>
<dispatcher>REQUEST</dispatcher>
</filter-mapping>
Restart Tomcat service
Access the resources using an iFrame.
The only question that has a bunch of answers. WElcome to the guide i wish i had when i was scrambling for this to make it work at 10:30 at night on the deadline day... FB does some weird things with canvas apps, and well, you've been warned. If youa re still here and you have a Rails app that will appear behind a Facebook Canvas, then you will need:
Gemfile:
gem "rack-facebook-signed-request", :git => 'git://github.com/cmer/rack-facebook-signed-request.git'
config/facebook.yml
facebook:
key: "123123123123"
secret: "123123123123123123secret12312"
config/application.rb
config.middleware.use Rack::Facebook::SignedRequest, app_id: "123123123123", secret: "123123123123123123secret12312", inject_facebook: false
config/initializers/omniauth.rb
OmniAuth.config.logger = Rails.logger
SERVICES = YAML.load(File.open("#{::Rails.root}/config/oauth.yml").read)
Rails.application.config.middleware.use OmniAuth::Builder do
provider :facebook, SERVICES['facebook']['key'], SERVICES['facebook']['secret'], iframe: true
end
application_controller.rb
before_filter :add_xframe
def add_xframe
headers['X-Frame-Options'] = 'GOFORIT'
end
You need a controller to call from Facebook's canvas settings, i used /canvas/ and made the route go the main SiteController for this app:
class SiteController < ApplicationController
def index
#user = User.new
end
def canvas
redirect_to '/auth/failure' if request.params['error'] == 'access_denied'
url = params['code'] ? "/auth/facebook?signed_request=#{params['signed_request']}&state=canvas" : "/login"
redirect_to url
end
def login
end
end
login.html.erb
&lt% content_for :javascript do %>
var oauth_url = 'https://www.facebook.com/dialog/oauth/';
oauth_url += '?client_id=471466299609256';
oauth_url += '&redirect_uri=' + encodeURIComponent('https://apps.facebook.com/wellbeingtracker/');
oauth_url += '&scope=email,status_update,publish_stream';
console.log(oauth_url);
top.location.href = oauth_url;
&lt% end %>
Sources
The config i think came from omniauth's example.
The gem file (which is key!!!) came from: slideshare things i learned...
This stack question had the whole Xframe angle, so you'll get a blank space, if
you don't put this header in the app controller.
And my man #rafmagana wrote this heroku guide, which now you can adopt for rails with this answer and the shoulders of giants in which you walk with.
The only real answer, if you don't control the headers on your source you want in your iframe, is to proxy it. Have a server act as a client, receive the source, strip the problematic headers, add CORS if needed, and then ping your own server.
There is one other answer explaining how to write such a proxy. It isn't difficult, but I was sure someone had to have done this before. It was just difficult to find it, for some reason.
I finally did find some sources:
https://github.com/Rob--W/cors-anywhere/#documentation
^ preferred. If you need rare usage, I think you can just use his heroku app. Otherwise, it's code to run it yourself on your own server. Note sure what the limits are.
whateverorigin.org
^ second choice, but quite old. supposedly newer choice in python: https://github.com/Eiledon/alloworigin
then there's the third choice:
http://anyorigin.com/
Which seems to allow a little free usage, but will put you on a public shame list if you don't pay and use some unspecified amount, which you can only be removed from if you pay the fee...
<form target="_parent" ... />
Using Kevin Vella's idea, I tried using the above on the form element made by PayPal's button generator. Worked for me so that Paypal does not open in a new browser window/tab.
Update
Here's an example:
Generating a button as of today (01-19-2021), PayPal automatically includes target="_top" on the form element, but if that doesn't work for your context, try a different target value. I suggest _parent -- at least that worked when I was using this PayPal button.
See Form Target Values for more info.
<form action="https://www.paypal.com/cgi-bin/webscr" method="post" target="_parent">
<input type="hidden" name="cmd" value="_xclick">
<input type="hidden" name="business" value="name#email.com">
<input type="hidden" name="lc" value="US">
<input type="hidden" name="button_subtype" value="services">
<input type="hidden" name="no_note" value="0">
<input type="hidden" name="currency_code" value="USD">
<input type="hidden" name="bn" value="PP-BuyNowBF:btn_buynowCC_LG.gif:NonHostedGuest">
<input type="image" src="https://www.paypalobjects.com/en_US/i/btn/btn_buynowCC_LG.gif" border="0" name="submit" alt="PayPal - The safer, easier way to pay online!">
<img alt="" border="0" src="https://www.paypalobjects.com/en_US/i/scr/pixel.gif" width="1" height="1">
</form>
I'm not sure how relevant it is, but I built a work-around to this. On my site, I wanted to display link in a modal window that contained an iframe which loads the URL.
What I did is, I linked the click event of the link to this javascript function. All this does is make a request to a PHP file that checks the URL headers for X-FRAME-Options before deciding whether to load the URL within the modal window or to redirect.
Here's the function:
function opentheater(link, title){
$.get( "url_origin_helper.php?url="+encodeURIComponent(link), function( data ) {
if(data == "ya"){
$(".modal-title").html("<h3 style='color:480060;'>"+title+" <small>"+link+"</small></h3>");
$("#linkcontent").attr("src", link);
$("#myModal").modal("show");
}
else{
window.location.href = link;
//alert(data);
}
});
}
Here's the PHP file code that checks for it:
<?php
$url = rawurldecode($_REQUEST['url']);
$header = get_headers($url, 1);
if(array_key_exists("X-Frame-Options", $header)){
echo "nein";
}
else{
echo "ya";
}
?>
Hope this helps.
I came across this issue when running a wordpress web site. I tried all sorts of things to fix it and wasn't sure how, ultimately the issue was because I was using DNS forwarding with masking, and the links to external sites were not being addressed properly. i.e. my site was hosted at http://123.456.789/index.html but was masked to run at http://somewebSite.com/index.html. When i entered http://123.456.789/index.html in the browser clicking on those same links resulted in no X-frame-origins issues in the JS console, but running http://somewebSite.com/index.html did. In order to properly mask you must add your host's DNS name servers to your domain service, i.e. godaddy.com should have name servers of example, ns1.digitalocean.com, ns2.digitalocean.com, ns3.digitalocean.com, if you were using digitalocean.com as your hosting service.
It's surprising that no one here has ever mentioned Apache server's settings (*.conf files) or .htaccess file itself as being a cause of this error. Search through your .htaccess or Apache configuration files, making sure that you don't have the following set to DENY:
Header always set X-Frame-Options DENY
Changing it to SAMEORIGIN, makes things work as expected:
Header always set X-Frame-Options SAMEORIGIN
i had this problem, and resolved it editing httd.conf
<IfModule headers_module>
<IfVersion >= 2.4.7 >
Header always setifempty X-Frame-Options GOFORIT
</IfVersion>
<IfVersion < 2.4.7 >
Header always merge X-Frame-Options GOFORIT
</IfVersion>
</IfModule>
i changed SAMEORIGIN to GOFORIT
and restarted server
Site owners use the X-Frame-Options response header so that their website cannot be opened in an Iframe. This helps to secure the users against clickjacking attack
There are a couple of approaches that you can try if you want to disable X-Frame-Options on your own machine.
Configuration at Server-Side
If you own the server or can work with the site owner then you can ask to set up a configuration to not send the Iframe buster response headers based on certain conditions. Conditions could be an additional request header or a parameter in the URL.
For example - The site owner can add an additional code to not send Iframe buster headers when the site is opened with ?in_debug_mode=true query param.
Use Browser extension like Requestly to remove response headers
You can use any browser extension like Requestly which allows you to modify the request & response headers. Here's a Requestly blog that explains how to embed sites in Iframe by bypassing Iframe buster headers.
Configure a Pass-through Proxy and remove headers from it
If you need to bypass Iframe buster headers for multiple folks, then you can also configure a pass-through proxy that just removes the frame buster response headers and return back the response. This is however a lot complicated to write, set up. There are some other challenges like authentication etc with the sites opened in Iframe through a proxy but this approach can work for simple sites pretty well.
PS - I have built both solutions and have first-hand experience with both.
Edit .htaccess if you want to remove X-Frame-Options from an entire directory.
And add the line: Header always unset X-Frame-Options
[contents from: Overcoming "Display forbidden by X-Frame-Options"
Use this line given below instead of header() function.
echo "<script>window.top.location = 'https://apps.facebook.com/yourappnamespace/';</script>";
Try this thing, i dont think anyone suggested this in the Topic, this will resolve like 70% of your issue, for some other pages, you have to scrap, i have the full solution but not for public,
ADD below to your iframe
sandbox="allow-same-origin allow-scripts allow-popups allow-forms"

How do I generate SEO-friendly markup for a single-page web app? [duplicate]

There are a lot of cool tools for making powerful "single-page" JavaScript websites nowadays. In my opinion, this is done right by letting the server act as an API (and nothing more) and letting the client handle all of the HTML generation stuff. The problem with this "pattern" is the lack of search engine support. I can think of two solutions:
When the user enters the website, let the server render the page exactly as the client would upon navigation. So if I go to http://example.com/my_path directly the server would render the same thing as the client would if I go to /my_path through pushState.
Let the server provide a special website only for the search engine bots. If a normal user visits http://example.com/my_path the server should give him a JavaScript heavy version of the website. But if the Google bot visits, the server should give it some minimal HTML with the content I want Google to index.
The first solution is discussed further here. I have been working on a website doing this and it's not a very nice experience. It's not DRY and in my case I had to use two different template engines for the client and the server.
I think I have seen the second solution for some good ol' Flash websites. I like this approach much more than the first one and with the right tool on the server it could be done quite painlessly.
So what I'm really wondering is the following:
Can you think of any better solution?
What are the disadvantages with the second solution? If Google in some way finds out that I'm not serving the exact same content for the Google bot as a regular user, would I then be punished in the search results?
While #2 might be "easier" for you as a developer, it only provides search engine crawling. And yes, if Google finds out your serving different content, you might be penalized (I'm not an expert on that, but I have heard of it happening).
Both SEO and accessibility (not just for disabled person, but accessibility via mobile devices, touch screen devices, and other non-standard computing / internet enabled platforms) both have a similar underlying philosophy: semantically rich markup that is "accessible" (i.e. can be accessed, viewed, read, processed, or otherwise used) to all these different browsers. A screen reader, a search engine crawler or a user with JavaScript enabled, should all be able to use/index/understand your site's core functionality without issue.
pushState does not add to this burden, in my experience. It only brings what used to be an afterthought and "if we have time" to the forefront of web development.
What your describe in option #1 is usually the best way to go - but, like other accessibility and SEO issues, doing this with pushState in a JavaScript-heavy app requires up-front planning or it will become a significant burden. It should be baked in to the page and application architecture from the start - retrofitting is painful and will cause more duplication than is necessary.
I've been working with pushState and SEO recently for a couple of different application, and I found what I think is a good approach. It basically follows your item #1, but accounts for not duplicating html / templates.
Most of the info can be found in these two blog posts:
http://lostechies.com/derickbailey/2011/09/06/test-driving-backbone-views-with-jquery-templates-the-jasmine-gem-and-jasmine-jquery/
and
http://lostechies.com/derickbailey/2011/06/22/rendering-a-rails-partial-as-a-jquery-template/
The gist of it is that I use ERB or HAML templates (running Ruby on Rails, Sinatra, etc) for my server side render and to create the client side templates that Backbone can use, as well as for my Jasmine JavaScript specs. This cuts out the duplication of markup between the server side and the client side.
From there, you need to take a few additional steps to have your JavaScript work with the HTML that is rendered by the server - true progressive enhancement; taking the semantic markup that got delivered and enhancing it with JavaScript.
For example, i'm building an image gallery application with pushState. If you request /images/1 from the server, it will render the entire image gallery on the server and send all of the HTML, CSS and JavaScript down to your browser. If you have JavaScript disabled, it will work perfectly fine. Every action you take will request a different URL from the server and the server will render all of the markup for your browser. If you have JavaScript enabled, though, the JavaScript will pick up the already rendered HTML along with a few variables generated by the server and take over from there.
Here's an example:
<form id="foo">
Name: <input id="name"><button id="say">Say My Name!</button>
</form>
After the server renders this, the JavaScript would pick it up (using a Backbone.js view in this example)
FooView = Backbone.View.extend({
events: {
"change #name": "setName",
"click #say": "sayName"
},
setName: function(e){
var name = $(e.currentTarget).val();
this.model.set({name: name});
},
sayName: function(e){
e.preventDefault();
var name = this.model.get("name");
alert("Hello " + name);
},
render: function(){
// do some rendering here, for when this is just running JavaScript
}
});
$(function(){
var model = new MyModel();
var view = new FooView({
model: model,
el: $("#foo")
});
});
This is a very simple example, but I think it gets the point across.
When I instante the view after the page loads, I'm providing the existing content of the form that was rendered by the server, to the view instance as the el for the view. I am not calling render or having the view generate an el for me, when the first view is loaded. I have a render method available for after the view is up and running and the page is all JavaScript. This lets me re-render the view later if I need to.
Clicking the "Say My Name" button with JavaScript enabled will cause an alert box. Without JavaScript, it would post back to the server and the server could render the name to an html element somewhere.
Edit
Consider a more complex example, where you have a list that needs to be attached (from the comments below this)
Say you have a list of users in a <ul> tag. This list was rendered by the server when the browser made a request, and the result looks something like:
<ul id="user-list">
<li data-id="1">Bob
<li data-id="2">Mary
<li data-id="3">Frank
<li data-id="4">Jane
</ul>
Now you need to loop through this list and attach a Backbone view and model to each of the <li> items. With the use of the data-id attribute, you can find the model that each tag comes from easily. You'll then need a collection view and item view that is smart enough to attach itself to this html.
UserListView = Backbone.View.extend({
attach: function(){
this.el = $("#user-list");
this.$("li").each(function(index){
var userEl = $(this);
var id = userEl.attr("data-id");
var user = this.collection.get(id);
new UserView({
model: user,
el: userEl
});
});
}
});
UserView = Backbone.View.extend({
initialize: function(){
this.model.bind("change:name", this.updateName, this);
},
updateName: function(model, val){
this.el.text(val);
}
});
var userData = {...};
var userList = new UserCollection(userData);
var userListView = new UserListView({collection: userList});
userListView.attach();
In this example, the UserListView will loop through all of the <li> tags and attach a view object with the correct model for each one. it sets up an event handler for the model's name change event and updates the displayed text of the element when a change occurs.
This kind of process, to take the html that the server rendered and have my JavaScript take over and run it, is a great way to get things rolling for SEO, Accessibility, and pushState support.
Hope that helps.
I think you need this: http://code.google.com/web/ajaxcrawling/
You can also install a special backend that "renders" your page by running javascript on the server, and then serves that to google.
Combine both things and you have a solution without programming things twice. (As long as your app is fully controllable via anchor fragments.)
So, it seem that the main concern is being DRY
If you're using pushState have your server send the same exact code for all urls (that don't contain a file extension to serve images, etc.) "/mydir/myfile", "/myotherdir/myotherfile" or root "/" -- all requests receive the same exact code. You need to have some kind url rewrite engine. You can also serve a tiny bit of html and the rest can come from your CDN (using require.js to manage dependencies -- see https://stackoverflow.com/a/13813102/1595913).
(test the link's validity by converting the link to your url scheme and testing against existence of content by querying a static or a dynamic source. if it's not valid send a 404 response.)
When the request is not from a google bot, you just process normally.
If the request is from a google bot, you use phantom.js -- headless webkit browser ("A headless browser is simply a full-featured web browser with no visual interface.") to render html and javascript on the server and send the google bot the resulting html. As the bot parses the html it can hit your other "pushState" links /somepage on the server mylink, the server rewrites url to your application file, loads it in phantom.js and the resulting html is sent to the bot, and so on...
For your html I'm assuming you're using normal links with some kind of hijacking (e.g. using with backbone.js https://stackoverflow.com/a/9331734/1595913)
To avoid confusion with any links separate your api code that serves json into a separate subdomain, e.g. api.mysite.com
To improve performance you can pre-process your site pages for search engines ahead of time during off hours by creating static versions of the pages using the same mechanism with phantom.js and consequently serve the static pages to google bots. Preprocessing can be done with some simple app that can parse <a> tags. In this case handling 404 is easier since you can simply check for the existence of the static file with a name that contains url path.
If you use #! hash bang syntax for your site links a similar scenario applies, except that the rewrite url server engine would look out for _escaped_fragment_ in the url and would format the url to your url scheme.
There are a couple of integrations of node.js with phantom.js on github and you can use node.js as the web server to produce html output.
Here are a couple of examples using phantom.js for seo:
http://backbonetutorials.com/seo-for-single-page-apps/
http://thedigitalself.com/blog/seo-and-javascript-with-phantomjs-server-side-rendering
If you're using Rails, try poirot. It's a gem that makes it dead simple to reuse mustache or handlebars templates client and server side.
Create a file in your views like _some_thingy.html.mustache.
Render server side:
<%= render :partial => 'some_thingy', object: my_model %>
Put the template your head for client side use:
<%= template_include_tag 'some_thingy' %>
Rendre client side:
html = poirot.someThingy(my_model)
To take a slightly different angle, your second solution would be the correct one in terms of accessibility...you would be providing alternative content to users who cannot use javascript (those with screen readers, etc.).
This would automatically add the benefits of SEO and, in my opinion, would not be seen as a 'naughty' technique by Google.
Interesting. I have been searching around for viable solutions but it seems to be quite problematic.
I was actually leaning more towards your 2nd approach:
Let the server provide a special website only for the search engine
bots. If a normal user visits http://example.com/my_path the server
should give him a JavaScript heavy version of the website. But if the
Google bot visits, the server should give it some minimal HTML with
the content I want Google to index.
Here's my take on solving the problem. Although it is not confirmed to work, it might provide some insight or idea's for other developers.
Assume you're using a JS framework that supports "push state" functionality, and your backend framework is Ruby on Rails. You have a simple blog site and you would like search engines to index all your article index and show pages.
Let's say you have your routes set up like this:
resources :articles
match "*path", "main#index"
Ensure that every server-side controller renders the same template that your client-side framework requires to run (html/css/javascript/etc). If none of the controllers are matched in the request (in this example we only have a RESTful set of actions for the ArticlesController), then just match anything else and just render the template and let the client-side framework handle the routing. The only difference between hitting a controller and hitting the wildcard matcher would be the ability to render content based on the URL that was requested to JavaScript-disabled devices.
From what I understand it is a bad idea to render content that isn't visible to browsers. So when Google indexes it, people go through Google to visit a given page and there isn't any content, then you're probably going to be penalised. What comes to mind is that you render content in a div node that you display: none in CSS.
However, I'm pretty sure it doesn't matter if you simply do this:
<div id="no-js">
<h1><%= #article.title %></h1>
<p><%= #article.description %></p>
<p><%= #article.content %></p>
</div>
And then using JavaScript, which doesn't get run when a JavaScript-disabled device opens the page:
$("#no-js").remove() # jQuery
This way, for Google, and for anyone with JavaScript-disabled devices, they would see the raw/static content. So the content is physically there and is visible to anyone with JavaScript-disabled devices.
But, when a user visits the same page and actually has JavaScript enabled, the #no-js node will be removed so it doesn't clutter up your application. Then your client-side framework will handle the request through it's router and display what a user should see when JavaScript is enabled.
I think this might be a valid and fairly easy technique to use. Although that might depend on the complexity of your website/application.
Though, please correct me if it isn't. Just thought I'd share my thoughts.
Use NodeJS on the serverside, browserify your clientside code and route each http-request's(except for static http resources) uri through a serverside client to provide the first 'bootsnap'(a snapshot of the page it's state). Use something like jsdom to handle jquery dom-ops on the server. After the bootsnap returned, setup the websocket connection. Probably best to differentiate between a websocket client and a serverside client by making some kind of a wrapper connection on the clientside(serverside client can directly communicate with the server). I've been working on something like this: https://github.com/jvanveen/rnet/
Use Google Closure Template to render pages. It compiles to javascript or java, so it is easy to render the page either on the client or server side. On the first encounter with every client, render the html and add javascript as link in header. Crawler will read the html only but the browser will execute your script. All subsequent requests from the browser could be done in against the api to minimize the traffic.
This might help you : https://github.com/sharjeel619/SPA-SEO
Logic
A browser requests your single page application from the server,
which is going to be loaded from a single index.html file.
You program some intermediary server code which intercepts the client
request and differentiates whether the request came from a browser or
some social crawler bot.
If the request came from some crawler bot, make an API call to
your back-end server, gather the data you need, fill in that data to
html meta tags and return those tags in string format back to the
client.
If the request didn't come from some crawler bot, then simply
return the index.html file from the build or dist folder of your single page
application.

HTML / Javascript One Click Print (no dialogs)

Is it possible to have a print option that bypasses the print dialog?
I am working on a closed system and would like to be able to pre-define the print dialog settings; and process the print as soon as I click the button.
From what I am reading, the way to do this varies for each browser. For example, IE would use ActiveX. Chrome / Firefox would require extensions. Based on this, it appears I'll have to write an application in C++ that can handle parameters passed by the browser to auto print with proper formatting (for labels). Then i'll have to rewrite it as an extension for Chrome / Firefox. End result being that users on our closed system will have to download / install these features depending on which browser they use.
I'm hoping there is another way to go about this, but this task most likely violates browser security issues.
I ended up implementing a custom application that works very similar to the Nexus Mod Manager. I wrote a C# application that registers a custom Application URI Scheme. Here's how it works:
User clicks "Print" on the website.
Website links user to "CustomURL://Print/{ID}
Application is launched by windows via the custom uri scheme.
Application communicates with the pre-configured server to confirm the print request and in my case get the actual print command.
The application then uses the C# RawPrinterHelper class to send commands directly to the printer.
This approach required an initial download from the user, and a single security prompt from windows when launching the application the first time. I also implemented some Javascript magic to make it detect whether the print job was handled or not. If it wasn't it asks them to download the application.
I know this is a late reply, but here's a solution I'm using. I have only used this with IE, and have not tested it with any other browser.
This Sub Print blow effectively replaces the default print function.
<script language='VBScript'>
Sub Print()
OLECMDID_PRINT = 6
OLECMDEXECOPT_DONTPROMPTUSER = 2
OLECMDEXECOPT_PROMPTUSER = 1
call WB.ExecWB(OLECMDID_PRINT, OLECMDEXECOPT_DONTPROMPTUSER,1)
End Sub
document.write "<object ID='WB' WIDTH=0 HEIGHT=0 CLASSID='CLSID:8856F961-340A-11D0-A96B-00C04FD705A2'></object>"
</script>
Then use Javascript's window.print(); ties to a hyperlink or a button to execute the print command.
If you want to automatically print when the page loads, then put the code below near tag.
<script type="text/javascript">
window.onload=function(){self.print();}
</script>
I am writing this answer for firefox browser.
Open File > Page Setup
Make all the headers and footers blank
Set the margins to 0 (zero)
In the address bar of Firefox, type about:config
Search for print.always_print_silent and double click it
Change it from false to true
This lets you skip the Print pop up box that comes up, as well as skipping the step where you have to click OK, automatically printing the right sized slip.
If print.always_print_silent does not come up
Right click on a blank area of the preference window
Select new > Boolean
Enter "print.always_print_silent" as the name (without quotes)
Click OK
Select true for the value
You may also want to check what is listed for print.print_printer
You may have to choose Generic/Text Only (or whatever your receipt printer might be named)
The general answer is: NO you cannot do this in the general case but there some cases where you might do it.
Check
http://justtalkaboutweb.com/2008/05/09/javascript-print-bypass-printer-dialog-in-ie-and-firefox/
If you where allowed to do such a thing anyway, it would be a security issue since a malware script could silently sent printing jobs to visitor's printer.
I found a awesome plugin by Firefox which solve this issue. try seamless printing plugin of firefox which will print something from a web application without showing a print dialog.
Open Firefox
Search addon name seamless printing and install it
After successful installation the printing window will get bypassed when user wants to print anything.
I was able to solve the problem with this library: html2pdf.js (https://github.com/eKoopmans/html2pdf.js)
Considering that you have access to it, you could do something like that (taken from the github repository):
var element = document.getElementById('element-to-print');
html2pdf(element);

iframe not displaying the html page

Inside the iframe tag - a CGI script is called - which return's HTML but which is not displayed. Even viewing the iframe -> view source gives nothing but empty HTML page.
While invoking the same CGI script directly from browser it works properly. It shows a simple html page with a single table.
What are all the likely issues ? Can somebody help ?
Issue in real context:
It is a reporting script from bugzilla 3.6 which is called in IFRAME which shows report properly in iframe. But the same script in bugzilla 4.0 which is called is not showing anything only in iframe.
I executed both scripts using wget - both gives exactly same html ( except some table fields - which cannot be an issue).
Direct cgi execution from browser of both scripts - and save page as html - both are same ( except some table fields - which cannot be an issue).
Tried debugging the cgi script - both prints almost same messages - some arguments - dumper and like ...
I have no idea of what else to check, please help. Thanks.
I came across the same issue and found the following in the Bugzilla 4.0 release notes
Bugzilla now sends the X-Frame-Options: SAMEORIGIN header with every
page request in order to prevent "clickjacking" attacks. Note that
this prevents other domains from displaying Bugzilla in an HTML frame.
Comment out the following lines in CGI.pm to prevent Bugzilla from setting the header.
# Add X-Frame-Options header to prevent framing and subsequent
# possible clickjacking problems.
unless ($self->url_is_attachment_base) {
unshift(#_, '-x_frame_options' => 'SAMEORIGIN');
}

How do I make Firefox auto-refresh on file change?

Does anyone know of an extension for Firefox, or a script or some other mechanism, that can monitor one or more local files. Firefox would auto-refresh or otherwise update its canvas when it detected a change (of timestamp) in the files(s).
For editing CSS, it would be ideal if just the CSS could be reloaded, rather than a full HTML re-render.
Effectively it would enable similar behaviour to Firebug with its dynamic HTML/CSS editing, only through external files.
Live.js
From the website:
How?
Just include Live.js and it will monitor the current page including local CSS and Javascript by sending consecutive HEAD requests to the server. Changes to CSS will be applied dynamically and HTML or Javascript changes will reload the page. Try it!
Where?
Live.js works in Firefox, Chrome, Safari, Opera and IE6+ until proven otherwise. Live.js is independent of the development framework or language you use, whether it be Ruby, Handcraft, Python, Django, NET, Java, Php, Drupal, Joomla or what-have-you.
It has the huge benefit of working with IETester, dynamically refreshing each open IE tab.
Try it out by adding the following to your <head>
<script type="text/javascript" src="http://livejs.com/live.js"></script>
Have a look at FileWatcher extension:
https://addons.mozilla.org/en-US/firefox/addon/filewatcher/
it's a WebExtension, so it works with the latest Firefox
it has a native app (to be installed locally) that monitors watched files for changes using native OS calls (no polling!) and notifies the WebExtension to let it reload the web page
reload is driven by rules: a rule contains the page URL (with regular expression support) and its included/excluded local source files
open source: https://github.com/coolsoft-ita/filewatcher
DISCLAIMER: I'm the author of the extension ;)
I would recommend livejs
But it has following Advantages and Disadvantages
Advantages:
1. Easy setup
2. Works seamlessly on different browsers (Live.js works in Firefox, Chrome, Safari, Opera and IE6+)
3. Don't add irritating interval for refreshing browser specially when you want to debug along with designing
4. Only refreshing when you save change ctrl + S
5. Directly saves CSS etc from firebug I have not used that feature but read on their site http://livejs.com/ that they support it too!!!
Disadvantages:
1. It will not work on file protocol file:///C:/Users/Admin/Desktop/livejs/live.html
2. You need to have server to run it like http://localhost
3. You have to remove it while deploying on staging/production
4. Doesn't serves CDN I have tried cheating & applying direct link http://livejs.com/live.js but it will not work you have to download and keep on local to work.
Xrefresh with firebug.
Firefox has an extension called mozRepl.
Emacs can plug into this, with moz-reload-on-save-mode.
when it's set up, saving the file forces a refresh of the browser window.
There are some IDE's that contain this ability (They'll have a pane within them or some other means to auto-refresh a page on save).
If you want to do this yourself a quick hack is to set the meta refresh on the page to a low value - one or two seconds.
# Will refresh the page content every second
<meta http-equiv="refresh" content="1" />
You could just place a javascript interval on your page, have it query a local script which checks the last date modified of the css file, and refreshes it if it changed.
jQuery Example:
var modTime = 0;
setInterval(function(){
$.post("isModified.php", {"file":"main.css", "time":modTime}, function(rst) {
if (rst.time != modTime) {
modTime = rst.time;
// reload style tag
$("head link[rel='stylesheet']:eq(0)").remove();
$("head").prepend($(document.createElement("link")).attr({
"rel":"stylesheet",
"href":"http://sstatic.net/mso/all.css?v=4372"
})
);
}
});
}, 5000);
Browsersync can do this from the server side / outside of the browser.
This can achieve more repeatable results / things that don't require so much clicking.
This will serve a page and refresh on change
cd static_content
browser-sync start --server --files .
It also allows a scripting mode.
This is certainly hacky, but if you want to work locally without making any external request (to live.js, for example), or run any local server, I think this might be useful. This is not specific to web development, you can adopt similar strategy to any other workflow.
You will need two tiny tools (which are present in almost all distribution repos): inotify-tools and xdotool.
First get the ID of your Firefox and your editor window using xdotool.
$ xdotool search --name "Mozilla Firefox"
60817411
60817836
$ xdotool search --name "Pluma" # Pluma is my editor
94371842
Depending on the number of processes running, you will get one or more window ID. Use xdotool windowactivate <ID> to know which one you want (the focus changes to the respective window).
Use inotifywait -e close_write to monitor changes to your local file and when you save the file using your editor, change focus to your browser, reload xdotool key CTRL+R and focus back to your editor. This is so instantaneous you will not notice nothing.
Also, inotifywait exits on change, so you might have to do it in a loop. Here is a minimum working example (in Bash in your working directory).
while /usr/bin/true
do
inotifywait -e close_write index.html;
xdotool windowactivate 60917411; # Switch to Firefox
xdotool key CTRL+R; # Reload Firefox
xdotool windowactivate 94371842 # Switch back to Pluma
done
You can use inotifywait to watch for the entire directory or some selected files in your directory.
You can write a script that can automate is easily.
This works on Linux (I've tested this on Void Linux.)
You can use live.js with a tampermonkey script to avoid having to include https://livejs.com/live.js in your HTML file.
// ==UserScript==
// #name Auto reload
// #author weirane
// #version 0.1
// #match http://127.0.0.1/*
// #grant none
// ==/UserScript==
(function() {
'use strict';
if (Number(window.location.port) === 8000) {
const script = document.createElement('script');
script.src = 'https://livejs.com/live.js';
document.body.appendChild(script);
}
})();
With this tampermonkey script, the live.js script will be automatically inserted to pages whose address matches http://127.0.0.1:8000/*. You can change the port according to your need.
I think that you can solve it by using some ajax requests after a determinate interval. You can do a request to CSS files and then if you don't get the "not modified" header you delete your css and load it again. For dynamic files you do a request and store the response and then every time you make a request to that file you compare the response to the latest.