Using PHP, how can I get the full HTML for a page after all Ajax requests have been executed on that page?
Basically, I would like to have the same HTML you see when you inspect an element in Google Chrome showing both the original HTML plus the extra markup added after the Ajax calls are done.
You'd need to use a headless browser to render the page. Check out Mink, as it appears to do what you're looking for.
Related
How is it possible for a web application/website to have the same identical source code behind every page on the site with no change at all. I was looking at the crypto - fiat currency exchange website: (Remitano.com) and for each page on the site the code is the same. How could this be possible?
The site is most likely a single page application (SPA). This means that it dynamically adds and removes elements with JavaScript. This is why sites made with React, Vue, and Angular don't work without JavaScript enabled.
For example, if you look at the code of a Vue app, you'll see something like <noscript><strong>We're sorry but myvueapp doesn't work properly without JavaScript enabled. Please enable it to continue.</strong></noscript>. The content inside noscript only displays if the user has JS disabled in their browser.
so I've been web scraping with Cheerio and I'm able to find the particular HTML element that I'm looking for, but for some reason, the text is not there.
For example in my web browser, when I inspect element I see Why Him?.
But, when I print out the object while scraping I see, so when I call the .text() function, it doesn't return anything. Why does this happen?
Inspect Element is not a valid test that Cheerio will be able to see something. You must use View Source instead.
Inspect Element is a live view of how the browser has rendered an element after applying all of the various technologies that exist in a browser, including CSS and JavaScript. View Source, on the other hand, is the raw code that the server sent to the browser, which you can generally expect to be the same as what Cheerio will receive. That is, assuming you ensure the HTTP headers are identical, particularly the ones relevant to content negotiation.
It is important to understand that while Cheerio is a DOM parser, it does not simulate a browser. So if the text is added via JavaScript, for example, then the text will not be there because that JavaScript will not have run.
If browser simulation is important to you, you should look into using PhantomJS. If you need a highly realistic browser rendering setup, then look into WebDriver and Leadfoot.
I am running a node.js server, and it is rendering a web page wonderfully. When I look at this in a browser, it runs exactly as I expect.
However, what I actually want to do is make the call to fully generate the html page - exactly as it is in the browser - within the node.js code, as a call. Currently, I have tried this:
http.request("http://localhost:8000/").end();
(with a few variants). This does exactly what it says, which is to make the single call to the server for the page - what it doesn't do is actually render the page, pulling in all of the other script files, and running the code on the page.
I have tried exploring express and ejs, and I think I need to use one of these, but I cannot find out how to do this fairly straightforward task. All it needs is to render an html page, but it seems to be a whole lot more complex than it should be.
What output do you want? A string of HTML? Maybe you want PhantomJS the headless browser. You could use it to render the page, then get the rendered DOM as a string of HTML.
Use the Mikeal's Request module to make http requests once you captured the response you then can inspect the html however you like.
To make that easier though you should use cheerio, this will give you a jQuery style api to manipulate the html.
Perhaps you are looking for wkhtmltopdf?
In a nutshell, it will render an entire web page (including images and JavaScript) to a PDF document.
I have a form that I have implemented using HtmlService. When I submit it I want to see different Html page instead of the page with the form on it. Basically this new page should replace the form page. How do I go about doing this. I tried to create a template form from the process form function that gets called when the form is submitted. But it didn't work. Help me out with this please.
See this answer for an example of serving multiple html pages using HtmlService. The basic idea is to write doGet() to accept a query parameter that it will use to select which html page to serve.
I am not aware of any server-side redirect mechanism (that you could use in the template, that is). However, one way how to do it is via Ajax, client-side. See, for example, how I did it in VALET (open index.html and look from line 240 on).
As an aside: I also tried reloading a page which seemed not to work (maybe due to the restrictions re the window and document object.
You can use doPost() with HTML Services to load another html page.
See this answer.
i'm trying to create a safari extension and i'm stuck with something.
I have my global.html on my extension and i've been searching the web for like 3hours on how to retrieve the full html code without success.
I tried:
document.getElementsByTagName('html')[0].innerHTML (when i alert i only see the code that i have inside my global.html)
document.documentElement.innerHTML (nothing happens)
etc...
I used safari.application.activeBrowserWindow.activeTab.url to retrieve the url but i cant get the html code.
Any help?
Thank you
You can't get the HTML of a web page from your extension's global page. You have to use an injected script and then, if you want, you can pass the web page's HTML to your global page using safari.self.tab.dispatchMessage. See this chapter in Apple's documentation.