I am trying to scrap some website using nodejs and puppeteer. Some element in the website is a bootstrap dropdown. And, I cannot get the content of my dropdown.
When I click view source code, the code displayed is not the same as when I inspect the page in the browser.
I understood that view source code load the code as it is rendered by the server, and inspect is showing source code after executing scripts.
Using puppeteer, the page is not loaded just like view source code.
So, what can I do to get the same source code as inspect ? Any help please ?
In the image below is the code when I do inspect:
With puppeteer or I click view source code, I don't get the li elements, just the ul element empty.
Related
I've got a simple MVC web page that pops up a dialogue box with a list of hyperlinks to files. They're properly prefixed with "file://" and the links work properly if copied to the clipboard and pasted into a browser window. However, from within the dialogue, clicking on the linked files returns... nothing.
Nothing at all happens. The behavior is identical in Firefox, Internet Explorer, and Chrome. No warnings, errors, etc.
Visually my dialogue looks like this:
If I "inspect element" on one of the links, for example, the top one for "javascript notes.txt", it looks like this:
This IS a valid hyperlink. As mentioned above, I can copy the link into the address bar of a browser and the linked file opens fine. I can also copy the full HTML of the element into notepad, wrapped in tags and save as an .html file, and the link works just fine from there. The links just don't work in the dialogue--from any browser.
I'm displaying the links in a Kendo grid currently. Thinking that might be a problem, I got rid of the grid temporarily and tried displaying them in a plain HTML . Same problem--clicking on the links produces no result at all.
Am I fighting something deeper here? Like, I'm using a jquery dialogue to show the list. do jquery dialogues do something to block hyperlinks from working?
TIA for any help.
As per Amy's comment on the original question, browsers seem to block the file:// links when opening from a page retrieved via http. The workaround I implemented came from the excellent article linked below: I coded a simple action method in a controller that returns not a view but a File, and changed the hyperlinks to invoke that controller method via http. Works like a charm.
see: http://rachelappel.com/upload-and-download-files-using-asp-net-mvc/
I want to get inspect element of the page web . I've tried Jsoup to get html but it only returns the source code. So is there a way to get the code fully like the one in the Inspect elements? or is there a way to get code from Inspect Elements? Thanks !
First of all I recommend using Chrome. Then use Inspect Element to get the code. If you want to actually download all of the source code, you should just go to File -> Save Page As... to get all the code.
Save the code to your desktop and you'll have everything you need to open the site locally in your browser.
I was going through a webpage and searched Google on how to find the page information.
I installed Firebug and there I found two things. The first is the HTML option and the second the Script option.
What is the difference between these two things? When I am looking at the page source it is exactly matching with the Script content shown in the Firebug. Then what is the code shown in the HTML option?
HTML Panel shows you the generated source of the page. This content can be different from what you see when you do a view source of the page. This pages shows you the DOM of the page. The Script Panel is something used to debug javaScript that generates the DOM. You can set break points and see the call stack on the right hand side pane of the Script panel. While as Right hand side panel of HTML block shows the CSS rules used to style DOM elements.
These "options" are actually called "panels".
The HTML panel allows you to inspect and change the HTML structure of the page. It is a live view of what is displayed on the page and provides you with different information related to the selected HTML element like applying CSS or its DOM properties. You can read more about the HTML panel here:
https://getfirebug.com/wiki/index.php/HTML_Panel
In contrary the code you get via the page's context menu option View Source is just a static view of the code when the page was requested from the server.
The Script panel is used for JavaScript debugging. It allows you to set breakpoints to stop the script execution at them. When the script is halted you can view the stack trace and the variables available in the current stack frame. You can read more about the Script panel here:
https://getfirebug.com/wiki/index.php/Script_Panel
What would cause an HTML table and form (viewable in view:source) not to render or appear on the page?
It also does not appear in Chrome's Developer Tools console or Firebug console.
I have been trying to figure this out all day and searching is not helping.
Any changes from the raw html you see in view source to the rendered DOM in the browser is caused by JavaScript.
There must be some code on that page that is removing your table.
Can you please provide some code examples of scripts that are on the page and that you are including?
I am trying to access the HTML code source of AJAX dynamically loaded content. How could I do it?
For example on Gmail, I am trying to access the HTML code of a given email discussion's content (the different entries of a given email discussion) which is loaded only when I click on this email discussion's line in the main list. The code source I can access is only the one of the page initially loaded (the list of all email discussions). Any idea?
Right-click on the page and select "Inspect Element". The element view is updated when JavaScript makes changes to the page, whereas the "View Source" view only shows content from when the page was loaded.
If you right click -> "View Source", it will show you the contents of a reloaded version of the page you are on.
Using "Inspect element" (hotkey CTRL+SHIFT+i) in Chrome shows the source of the dynamic content.
I think you want to bind event on the dom element that loaded after initial page load using ajax. If so then you can use jQuery library and you can use live method of jquery.
Here is a link for live method, you can check that.
http://api.jquery.com/live/
What you need
Download jquery latest library (http://docs.jquery.com/Downloading_jQuery)
and then include it in script within head
or you can use cdn http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js
then do like following
<script>
$(document).ready(function(){
$("#ajax_dom_element_id").live('click_or_any_event',function(){
code snippet you wan dot
});
});
</script>
I think it will be helpfull.