We're looking for a tool that would allow us to script and run tests across multiple browsers including Chrome, FF 3,4, IE 6,7,8 and during the running allow a human to visually inspect each page / action. This must support javascript so a simple browsershots approach will not work. Something like Selenium Grid may work but I haven't seen anything that would act like a console so that a human could visually inspect each test step. Ideally the answer to this question would be a tool that runs a scripted front-end test and displays those tests on each browser in a grid so that a human could visually inspect the look of each page while the test was testing the html content.
How about Cucumber Sauce?
I've not tried it purely because I do not need to run tests in parallel across multiple browsers or headless. However, I am using Cucumber BDD for almost all my web application testing. Cucumber Sauce looks like it might have solved the parallelism problem so it might work for you.
How about Telerik Test Studio http://www.telerik.com/automated-testing-tools.aspx? It can do FF 3 & 4, IE7, 8, 9 (but not IE6), Chrome and Safari. It literally hooks into the browser and drives it as if a real person was sitting there clicking things and typing text.
A couple months ago I ran into a browser that allowed users to see how different rendering engines displayed their site: Lunascape, and here are its features. As far as it validity, or accuracy goes, I cannot vouch for it (I have never used it); I can link you to a CNET review though: CNET.
So my best idea would be to use lunascape (to allow your tester to see all the engines side-by-side), then script the tests in javascript which is included in every page.
Good luck, and hope this helps.
TestComplete can do this: http://www.automatedqa.com/products/testcomplete/
You can record any interaction with an application and when the tests run you can watch them.
It supports:
Internet Explorer 6—9 support (both 32- and 64-bit versions)
Firefox 3.5—4.0 support
Open source tool Selenium, which supports many browsers and can be run as a simple record / playback tool or from code but the browser compatibility varies a bit depending on which method you choose. Google uses Selenium in a big way these days and there are plenty of good articles such as this one on their testing blog.
Very good question.
The closet tool that i think of will be HP load runner for :
the ability to visually inspect each page while the test is running and inserting break point if needed
Javascript is supported but you might have problem with asynchronous request (ajax)
Since there is alway a but :
Not all browsers are simulated.
Browsers are simulated also ...
The report will print the performance and not what was shown on the page.
The price
I do not think that visual inspection falls into automated test. Automated test is about performance and to detect regression. Visual inspection on the other side is to check if the layout is properly rendered.
Doesn't this testing depend on the size of the site to be tested? If I have a 5-10 page site, I would use some of the tools mentioned above, handle it manually (i.e. walk through the site in each browser on each platform), use browsershots.org, or use a tool within Safari, such as simulating a browser.
If the site is large, then these options become very difficult and time consuming, and my experience is that the best thing to do is make sure that each page is standards compliant, and try different pages in a few browsers.
Hope this helps.
Related
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
My team is working on a new site which should be cross browser compatible (IE 8+, Chrome, Safari, Firefox, Opera) and cross platform compatible (Desktop, Tablet, Smartphone).
We've been looking at a lot of the new methodologies for achieving this such as HTML 5, responsive design, using JS libraries that abstract a lot of the browser mess away from the user but since the browsers and even MVVM, but the one major issue that I've been facing is how fast the browsers are changing. With both Chrome and Firefox using a model of continuous delivery it becomes harder and harder to test. And from the looks of it other sites have the same problem (it seems like it's hit or miss these days as to whether a site will work in a particular browser)
What suggestions do you or your team have for testing new browsers as they come out?
What things do you do during development to decrease possibility of having code break when a browser update comes out?
And how do you decide when you will or will drop support for a browser version?
NOTE: I've altered the order of your questions to leave the longer answer at the end.
Your questions
What suggestions do you or your team have for testing new browsers as they come out?
Actually, as you said, Chrome and Firefox are continuosly delivering so it eases the process. The last version you have is mostly always the version the user has.
For any other browser (and Chrome and Firefox old versions) just select a version of each and act as a "high pass filter", testing any version up from the one you selected.
How do you decide when you will or will drop support for a browser version?
Take a look at the statistics of browser use. There are many resources such as statcounter, w3counter, w3cschools, or wikimedia. If possible, add an analytics tracker to your page and you will have data about what devices, platforms, browsers, and versions of them the visitors use to access the site.
What things do you do during development to decrease possibility of having code break when a browser update comes out?
The key is to use a well defined methodology, based on the existing standards. Continue reading for a personal recommendation.
Workflow to ease cross-browsing
Step 1: Bootstrapping
At first decide: Graceful degredation versus progressive enhancement. Both are valid techniques, but makes sense using the first to fix existing projects and the second for newly created projects.
Then select libraries to avoid typing existing code, focusing on the 3 languages: JavaScript, CSS, and HTML. HTML5 (+CSS3) is the better choice today but support for older browsers must be provided. The following libraries ease supporting them:
modernizr for feature detection and conditional loading of js or css.
jQuery for ajax and dom related tasks.
normalize.css for normalize default browser styles, rather than just "resetting" them.
Notice that all of the js libraries listed above allow custom builds, an important thing when performance matters.
Html5 Boilerplate provides a strong template from which start the layout. It includes modernizr, jQuery, and normalize.css. Its github repository is a good resource to learn a lot about cross-browsing techniques. This article on its wiki has a nice set of links to start learning.
Step 2: Do the work
Designs should be mobile-first and responsive. This article on html5rocks introduces well why and how.
While "doing the work":
Follow the w3c standards. Avoids using hacks, specially CSS hacks, when possible. Review often the HTML5 specification as it is pretty unstable.
Take care of ECMAScript 5 features when writing javascript. Rely on libraries to avoid code breaks caused by deficient browser implementations. Do not extend the DOM.
Automate tests when possible. Layout and specially layout polishing, including animations, are manually tested cause it's quicker but UI functionalities like form submision can be perfectly tested with automated tests.
Use tools to ease tasks. Chrome + devtools or Firefox + firebug are the very basic must-use, but there are a bunch of tools to ease cross-browsing tests, even automating those tests.
Annex: Tools and resources
Cross-browser testing
Browserstack is just awesome. Allows testing on all devices, platforms, browsers, and versions.
Browserling is an alternative to browserstack. It is developed and maintained by Peteris Krumins and James Halliday, both recognized members of the node.js community and well-known developers. They also published a tool to automate the process called testling-ci, but this is only relevant if using node.js on the back-end.
modern.ie provides tools to ease testing on internet explorer. Developed by Microsoft, the site provides live testing through browserstack and downloadable virtual machine images with pre installed software.
adaptability testing for "responsive design"
respon.si is an online tool meant to test the visually appearance of layouts. It allows selecting a resolution so it's useful for responsive layouts testing. Notice that any other tool to select a resolution can easily do the same.
What suggestions do you or your team have for testing new browsers as they come out?
As part of our definition of done we support the following desktop browsers:
IE8+
Firefox 3.6
Firefox (latest)
Chrome (latest)
Safari 6
The support of the latest versions of Firefox/Chrome is fine because they both provide automatic updates, and so if anybody has a problem on an older version of the browser, it's out of our hands and they should update.
The majority of Firefox/Chrome testing can be done on our machines, but there are obviously discrepancies with how the different OS' handle fonts, and some quirks with native form elements that may or may not carry over to versions on Windows.
To test Firefox versions on OS X I use the "Install all Firefoxes" script that I created, to allow me to run multiple versions of Firefox side-by-side.
Our development team uses Ubuntu and Mac OS as their environments, so we have a dedicated machine with virtual machines for each version of IE, and Chrome/Firefox on Windows, and Safari 6 on OS X.
These virtual machines were setup using the images provided by modern.ie. We're remotely accessing the machine with the virtual machines on so that we don't need to break our workflow and go to another machine.
What things do you do during development to decrease possibility of having code break when a browser update comes out?
The obvious things are avoiding CSS hacks, and making sure that the HTML/CSS/JavaScript that is written meets our code standards, and our definition of done.
If we're using experimental CSS features, we ensure that we're providing vendor prefixed and lastly w3 definitions of properties:
-moz-foo: bar;
-ms-foo: bar;
-o-foo: bar;
-webkit-foo: bar;
foo: bar;
Obviously this introduces some technical debt, but if you're using a preprocessor for your CSS, the overhead of this can be reduced.
We keep a separate stylesheet for IE and load it using conditional comments, so that we can fix problems in IE without affecting the integrity of the rest of the front-end for other browsers. There's a movement lately to moving this in to a shame.css though, which you can read about here: http://csswizardry.com/2013/04/shame-css/
And how do you decide when you will or will drop support for a browser version?
Google Analytics. Segmenting by browser type in Google Analytics is very useful. When usage for a particular browser drops below 10% it's a good time to stop developing new features for it.
You could do something as radical as throwing away all of your hacks/styles for that browser too, but for a smoother transition (and to encourage people to upgrade) it's better to simply stop developing for the older browser, and maybe conditionally display a message.
We dropped support for IE7 recently, and now visitors using IE7 will get a message telling them to upgrade, and they don't get any more fixes or additional features.
Mobile is a whole other kettle of fish, and if your site is completely responsive it's an extra layer of pain.
We've got a bunch of different size/version Android devices, a couple of old iPhones and an iPad kicking around the office that we use for testing the majority of mobile browsers on.
Obviously there are differences with screen size, DPI, browser version etc. The best you can do in this regard is cater for the most common case, and fix any issues as they surface.
design and develop with web standards. validate the web platform. all of the libs/solutions/etc., already listed are phenomenal and quite useful, but if you stick to standards, you'll find you need those libs less and less...at least for gracefully degrading user agent support. i find i use them more and more for progressive enhancement.
What are the current best recommended resources for cross-browser testing in the CSS3 jquery world? I am adopting html5 and have adopted parts of css3 into my latest web project. I have also changed over to jquery for my scripting needs. The best answer I have seen for testing this pre-dates html5 and css3 being ready for primetime.
I am testing in Windows Opera, Safari, 2 Firefoxes+firebug, IE+f12 and two Androids. I test myself for input=expected output in my javascript and browser-sizing and other rendering issues. I test my php generated code by rendering it in firefox, then validating. I am trying to test for user experience by having myself and other people use the site and reporting on their opinion of the design, usability and overall impressions of the site-in-action. I present them with views on different sized devices. I even am lucky to have a friend with color-blindness;-} The F12 solution is particularly dissatisfying when testing in IE and the css rendering consumes large chunks of my development time.
Is better emulation available? Has anyone found a better (i.e. fast, easy, efficient, scientific) way to test across platforms yet? I am hoping for a strategy like unit-testing to emerge in the online community so we can make our apps more stable and robust as they become more powerful and influential. I can't seem to find a way to tame this chaos!
Have you used Selenium?
http://seleniumhq.org/
It is a tool to automate browser testing. Some people at Google made it and open sourced it.
You may want to look at RIATest for cross-platform cross-browser testing of HTML5 applications.
It works on Windows and Mac, supported browsers are Firefox, IE and Chrome. Automated testing scripts written on one platform/browser can be run against all other supported platforms/browsers.
(Disclaimer: I am a RIATest team member).
What would be the process for ensuring that the code works as expected across multiple browsers. What would be the best answer?
Be XHTML compliant (w3.org validator)
Be CSS compliant (w3.org validator)
Use a JavaScript library that is cross-browser compatible (less direct call to JavaScript as possible)
Test, test, test during development. Not at the end!
Avoid bleeding edge code.
Yeah, I know, many of you will hate that answer. And if you've never worked in a large enterprise environment, you'll think I'm a Luddite. However, I can't tell you how many times the requirements I've been given have specifically listed "No HTML5" or "No CSS3" elements simply because the client was paranoid about IE6 working exactly as the others did.
The obvious overall answer is testing, but I'd go one step further. If you're worried about 100% operation in all browsers, you have to define your standards. For example, if you have to code back to IE6, do you have to worry about mimicking rounded corners, which is always a hack on IE6 and below? Or, will the client accept progressive enhancement such as square corners for those on browers from the dark ages and rounded for the rest of us? Does the client specify fonts that can't be told apart when pulling the page up side-by-side, or do they understand that browsers use different font rendering engines? Is it ok to work just in IE6, or do you have to also support quirks mode? What about rendering with a screen reader (accessibility) or without CSS or Javascript. How about mobile devices? All these were valid and measurable issues with my last major corporate client.
I like Adobe's Browser Labs as my first line of defense for testing. However, it's just one of many steps I take including multiple physical computers on multiple OS's connecting via multiple connections through different network proxies. You just can't test enough....and even then expect to find an error as the site is launched and matures.
Take each browser and test :D
You can use websites like browsershots.org to see how it looks on different browsers and platforms.
The most comprehensive way of doing that is to actually test in different browsers. A simple solution would be to create a virtual machine hosting server, set up multiple VMs, and then install a different browser version to different VMs to fully test your application.
Absent that, there are tools that can emulate (but not fully) browsers and you can test through those.
The best measures to adopt are:
Use a CSS Reset (read about it at the link, implement it however you like)
Use a Javascript Framework like jQuery (This will abstract a lot of cross browser quirks)
Validate your HTML and CSS. Make sure you are specifying a Doctype
You can test most browsers via Chrome, Firefox and IE8. IE8 has an IE7 mode that can be used to test for IE7. Press F12 in IE8 to get the developer window to debug and solve issues. Make sure you are prompting for all JS errors. IE6 is a tough one, but there are a number of resources available that you can probably find to help you with this.
Hope this helps. Good luck.
http://browsershots.org/
thats the site you looking for. You need to put in the url of your website, select the browsers that you want to check and click submit. It will return screenshots of the your website based on the browsers that you have selected.
There is no one way to test the app's compatibility for web browsers. First thing to keep in mind is understand the standards set for the app, define the number of browsers and versions to support. Once we know what we need to support we can keep the following points to ensure compatibility:
Test during development. Not at the end.
Avoid bleeding edge code. New functions that come with ES5 or ES6 standards would only be supported by modern browsers, hence would need polyfills in older browsers. Therefore use the most native form of Javascript wherever possible.
Use jQuery functions if it's included in the project. It helps take care of most javascript cross browser issues. If not included, use just bring its particular function in your project that you might need.
For CSS, try to use the traditional methods of layout and styling instead of the latest CSS3 one's which might not be supported by old browsers(like transform property).
Tools like Browserstack can be used to see the screenshots of how the CSS turned out on different machines.
Actually test on different machines and browsers. Although chrome provides a superb emulator but when code actually runs on that particular OS and ecosystem, then it may misbehave. So the best way to ensure is actually test them in every ecosystem.
Use Tools like VirtualBox to be able to test old browsers and different OS.
There is no way to ensure it aside from testing testing testing :-)
I am working in creating a website and I want to check in multiple browsers for Browser Compatibility test automatically by using Automated Test Tool. Do you guys know if there is any software/tool where I can just give a link and it loads the page in multiple browsers?
I've used Browser Shots before and it's ok if you don't mind waiting for an hour or two.
I'd also recommend checking out some of the links on Delicious.
You're looking for litmus, from the people who brought you doctype, part of the League of Justice. 14-day passes to test your layouts in 24 browsers currently cost $39.
I'm a fan of XenoCode's "Spoon Browser Sandbox" myself.
You can use Selenium RC (Selenium 1) or Selenium 2 (WebDriver) for automated test. But You have to record the test using Selenium IDE on Firefox browser and writing some tests on it. Selenium supports FF, Chrome, IE, Safari and Opera.
If the website is publicly visible then there are web based services that you can use such as http://crossbrowsertesting.com/.
If your website is internal only, then you're going to struggle to find support I think. We tried to find one but all we found were services that require a publicly visible website - no good for testing pre-go-live.
Are you talking about a compatible design or compatible JS? Because AJAX functionality is difficult to test with the usual cross-browser tools.
For the latter, look at httpUnit, though I'm not sure it can simulate multiple browsers.
There's Microsoft Expression Web SuperPreview, but I haven't tried it so can't comment on if it's good or not.
Their opening quote makes me laugh though, mainly because of how sucky IE6 is:
About 7 years ago, the browser wars
were over. Internet Explorer had
become the de facto standard, and for
a while, there were very few
compatibility issues in web page
design.
Check it out though, might be worthwhile. Especially if you are a .NET developer, however it might be useful to web developers in general.
Adobe has BrowserLab. It requires an Adobe account (free) and gives you Firefox 2.0 - 3.5 (WinXP, OSX), IE 6-8 (WinXP), Safari 3-4 (OSX) and Chrome 3.0 (WinXP).
Is anyone writing applications specifically to take advantage of google chrome?
Are there any enterprise users who are considering using it as the standard browser?
Yes, I have started to pay very good attention to Google Chrome for my applications. Recent analytics show that between 6%-15% of my users are accessing my applications (varies between 6 to 15 in different applications) on Chrome. And, this number looks on an upward trend.
Thus, I can't really ignore it for testing right now.
As far as taking it as a standard goes, thats a long way off. I still have to test for IE6! :( Though, we have been planning to start using features like Gears (inbuilt in Chrome - downloadable elsewhere) once Chrome crosses the 25% mark. Thats when I believe that we will be looking at Chrome to be our preferred browser. I hope that we have Chrome 1.0+ by then! ;)
I switched to Chrome and haven't looked back except for the occasional site which doesn't work properly, forcing me to load it in Firefox. All my existing web applications work fine on it, and I'm using it for primary testing on my current development project.
I'm not actually targeting chrome, but I have added chrome to my browsers to test sites on. I've found some odd quirks in this product where some plugins cause the browser to hang, or run really slow in some environments, but they are still in beta in active development. But I definately now make sure sites I work on render well in chrome, as well as firefox, latest versions of IE, safari, Konquerer and opera. I usually check out how it looks on lynx as well, that helps me catch "un-alternated text" in images. Yeah, I know that isn't a word, but some people will understand what I'm saying.
Because chrome uses the webkit to render HTML, you can be assured if it works in safari, it'll work under chrome, however it's rendering engine isn't up to scratch quite yet. I think writing applications that take advantage of it is similar to writing iPhone applications, remember chrome is expected to be adopted by android to make it similar to iPhone. That way it pretty much takes advantage of all those iPhone apps.
Would I install it as the browser of choice? not yet - but i'll certainly work on valid web pages that will render across all browsers.
One of our major customers has outlawed Chrome because it installs on the C drive without asking. They deploy a standard image with a small C drive and large D drive so they can easily re-clone the system part of the image on C without destroying the client's personal files on D. Most software allows you to choose the install directory. Anything that violates this is disallowed, and they're a big enough company to have some weight with most vendors.
We have enough headaches trying to support
Firefox
Two versions of IE which have their own iffy bugs
Safari
I'm not sure why we continue to support Safari. Most of our users (corporate) use IE6 or IE7. We try to make sure that things work in both of those.
Maybe not for programming purposes but Chrome w/ Google Reader makes for the most powerful RSS reader. Can handle up to 1500 feeds w/ performance still ok, managing subscriptions still functioning.
I'm using it on my work machine, but that's about it. It's been stable for me, and I like the barebones UI. I'll still switch to Firefox for the web developer extensions however.
I'm liking some of GoogleChrome- the Start page with your 9 most recent is the winner for me. The interface takes a little getting used to, but the speed is impressive, especially with Gmail.
However, it glitches with Java, which rules it out for serious work at the moment. I use FireFox mostly and have Chrome for the "other" websites at work.
I'm considering using GWT on an intranet project and considering suggesting to the users that use Chrome to take advantage of the enhanced Javascript performance. Any AJAX-heavy app would be a great candidate to target Chrome.
At my company, we're not targeting it, but we're definitely paying attention to it. My boss is using it as his primary browser, and I have implemented browser detection for it in our scripts in case we ever to need to target it for some reason.
Chrome has the .png opacity bug where the transparent parts of the .png are a solid color if you try to transition the opacity from 0 to 1. In IE7 the opaque parts are black, and in Chrome, they are white. Today, I decided to go ahead and account for this bug in my JavaScript. I don't really test sites on Chrome that often, but I am actually using it for almost all of my browsing.
I will target Chrome as soon as a stable Linux and OSX client is available.
Targeting Chrome/Chromium right now, I think is like targeting Konqueror web browser. It will get popular, but you should wait to a more stable beta, and/or some Linux and OS X client.
My website statistics shows 3.xx % visitors using Chrome which arrived just few weeks back. And Opera is only 4.xx % which has been around for several years.
Easily you can see that rate at which Chrome is picking up.
You can see how easily Google takes over all areas of your computing world and personal world too.
Since Chrome uses Webkit, it has the same rendering engine and DOM support as Safari (not necessarily the same revision of Webkit though). By testing in Safari, you can generally get by without worrying about Chrome. Any differences you find are probably just bugs that you should file on instead of work around.
However, because Chrome uses a different JS engine, there may be a few incompatibilities with Safari. So, if you're doing anything with JS, you might as well fire up Chrome and see if there's anything obviously wrong.
Generally though, you don't target browsers, you target rendering engines (with their associated DOM support and JS engines).
I am using Google Chrome, so far all the web apps I have work fine in it with no modifications.
No.
Why help Google further build an evil empire? In this particular case it is so obvious that they do not care about users but only obsessed with gathering usage info.
It's not any major player yet