I have an application which is sending me JSON object. Now I want to see how this json data structure in javascript alert something like this. How can I do that?
You can't even develop in Chrome/firefox ? I would look at changing this, it would be like coding with one hand tied behind your back. My preferred method is using Chrome dev toolbar, but here are some other ways.
You can debug using a browser based debugger, for ie .
You could use the JSON.stringifiy and alert the output,the code is here
And there are these viewers as well http://jsonviewer.stack.hu/ and jollydroll
And finally you can loop through the different object properties like so :
for(var propertyName in yourJSON){
//will loop through the different elements in your json
alert(yourJSON[propertyName]); //will output the valueof each element
alert(propertyName); //will output name of each element
}
You could throw the JSON into Firebug's console (or Chrome's developer console?) and get a pretty-print view of it.
easiest would be to use alert(json) or console.log(json) - available with development tools
In IE8, Tools->Developer Tools (F12), Script.
Set a breakpoint and examine the JSON result in Locals? It won't be 'structured', but should be readable if it's not huge
Related
Im trying to scrape a div class but everything I have tried has failed so far :(
Im trying to scrape the element(s):
<a href="http://www.bellator.com/events/d306b5/bellator-newcastle-pitbull-vs-
scope"><div class="s_buttons_button s_buttons_buttonAlt
s_buttons_buttonSlashBack">More info</div></a>
from the website: http://www.bellator.com/events
I tried accessing the list of elements by doing
Elements elements = document.select("div[class=s_container] > li");
but that didnt return anything.
Then i tried accessing just the parent with
Elements elements = document.select("div[class=s_container]");
and that returned two div with classname "s_container", non of which is the one I needed :<
then i tried accessing that ones parent with
Elements elements = document.select("div[class=ent_m152_bellator module
ent_m152_bellator_V1_1_0 ent_m152]");
And that didnt return anything
I also tried
Elements elements = document.select("div[class=ent_m152_bellator]");
because I wasnt sure about the white spaces but it didnt return anything either
Then I tried accessing its parent by
Elements elements = document.select("div#t3_lc");
and that worked, but it returned an element containing
<div id="t3_lc">
<div class="triforce-module" id="t3_lc_promo1"></div>
</div>
which is kinda weird because i cant see that it has that child when i inspect the website in chrome :S
Anyone knows whats going on? I feel kinda lost..
What you see in your web browser is not what Jsoup sees. Disable JavaScript and refresh page to get what Jsoup gets OR press CTRL+U ("Show source", not "Inspect"!) in your browser to see original HTML document before JavaScript modifications. When you use your browser's debugger it shows final document after modifications so it's not not suitable for your needs.
It seems like whole "UPCOMING EVENTS" section is dynamically loaded by JavaScript.
Even more, this section is asynchronously loaded with AJAX. You can use your browsers debugger (Network tab) to see every possible request and response.
I found it but unfortunately all the data you need is returned as JSON so you're going to need another library to parse JSON.
That's not the end of the bad news and this case is more complicated. You could make direct request for the data:
http://www.bellator.com/feeds/ent_m152_bellator/V1_1_0/d10a728c-547e-4a6f-b140-7eecb67cff6b
but the URL seems random and few of these URLs (one per upcoming event?) are included inside JavaScript code in HTML.
My approach would be to get the URLs of these feeds with something like:
List<String> feedUrls = new ArrayList<>();
//select all the scripts
Elements scripts = document.select("script");
for(Element script: scripts){
if(script.text().contains("http://www.bellator.com/feeds/")){
// here use regexp to get all URLs from script.text() and add them to feedUrls
}
}
for(String feedUrl : feedUrls){
// iterate over feed URLs, download each of them
String json = Jsoup.connect(feedUrl).ignoreContentType(true).get().body().toString();
// here use JSON parsing library to get the data you need
}
ALTERNATIVE approach would be to stop using Jsoup because of its limitations and use Selenium Webdriver as it supports dynamic page modifications by JavaScript so you'd get the HTML of the final result - exactly what you see in web browser and Inspector.
If anyone finds this in the future; I managed to solve it with Selenium, dont know if its a good/correct solution but it seems to be working.
System.setProperty("webdriver.chrome.driver", "C:\\Users\\PC\\Desktop\\Chromedriver\\chromedriver.exe");
WebDriver driver = new ChromeDriver();
driver.get("http://www.bellator.com/events");
String html = driver.getPageSource();
Document doc = Jsoup.parse(html);
Elements elements = doc.select("ul.s_layouts_lineListAlt > li > a");
for(Element element : elements) {
System.out.println(element.attr("href"));
}
Output:
http://www.bellator.com/events/d306b5/bellator-newcastle-pitbull-vs-scope
http://www.bellator.com/events/ylcu8d/bellator-215-mitrione-vs-kharitonov
http://www.bellator.com/events/yk2djw/bellator-216-mvp-vs-daley
http://www.bellator.com/events/e8rdqs/bellator-217-gallagher-vs-graham
http://www.bellator.com/events/281wxq/bellator-218-sanchez-vs-grimshaw
http://www.bellator.com/events/8lcbdi/bellator-219-koreshkov-vs-larkin
http://www.bellator.com/events/9rqguc/bellator-macdonald-vs-fitch
Im trying to scrape data from this website: http://www.bundesliga.de/de/liga/tabelle/
In the source code i can see the tables but there's no content, just things like:
<td>[no content]</td>
<td>[no content]</td>
<td>[no content]</td>
<td>[no content]</td>
....
With firebug (F12 in Firefox) i wont see any content too but i can select the table and then copy the innerHTML via firebug option. In that case i get all the informations about the teams, but i dont know how to get the table with the content in Jsoup.
To get the value of an attribute, use the Node.attr(String key) method
For the text on an element (and its combined children), use Element.text()
For HTML, use Element.html(), or Node.outerHtml() as appropriate
For example:
String html = "<p>An <a href='http://example.com/'><b>example</b></a> link.</p>";
Document doc = Jsoup.parse(html);
Element link = doc.select("a").first();
String text = doc.body().text(); // "An example link"
String linkHref = link.attr("href"); // "http://example.com/"
String linkText = link.text(); // "example""
String linkOuterH = link.outerHtml();
// "<b>example</b>"
String linkInnerH = link.html(); // "<b>example</b>"
reference:
http://jsoup.org/cookbook/extracting-data/attributes-text-html
The table is not rendered on the server directly, but build by the client side JavaScript of the page and constructed with data that is getting to the client via AJAX. So what you get with the naive Jsoup approach is expected.
I see two possible solutions:
You analyze the network traffic and identify the ajax calls that the site is making. Then you try to reconstruct the format and fire the same requests as the JavaScript would. Then you can reconstruct the table.
you don't use Jsoup but a real browser, that loads the page and runs the JavaScript including all AJAX calls. You could use Selenium webdriver for that. There is a headless browser called phantomjs which has a relatively small footprint that you can use in combination with selenium webdriver.
both options have their (dis)advantages:
This takes more time, since you need to understand the network traffic pretty good. The reward will be a very fast and memory efficient scraper.
The programming of selenium is very easy and you should not have any difficulties achieving your goal. You don't need to understand the inner workings of the site you want to scrape. However, the price is a further dependency in your project. Memory consumption is high. Another process runs. The scraping will be slow.
Maybe you find another source with the soccer table that is holding the infos you want? That might be the easiest. For example http://www.fussballdaten.de/bundesliga/
I am using Google Chrome Developer Tools to try to see the response of some AJAX url's.
The problem is that when I click on the NETWORK TAB, then on the link, then on RESPONSE, I see this text : "THIS REQUEST HAS NO RESPONSE DATA AVAILABLE".
I have been using FIREBUG and I am 100% sure there is a response from that page.
Can somebody help with this ?
Thank you !
You can try manually checking if there's a response or not
So, generally when dealing with ajax, in most cases we use the POST, You can create a 'same structured' page to handle same input/response but using Get method and print the output data as normal.
This way you can see if there's any response/errors in your script very easily
I am trying to open HTML file from the local URI which I use as XML Editor, to edit xml data that come from Silverlight application, then close browser window and return back edited xml data to the Silverlight application.
I tried to use HtmlPage.Window.Navigate but I don't quit like it.
I have tried using a method from: http://weblogs.asp.net/dwahlin/archive/2010/05/10/integrating-html-into-silverlight-applications.aspx
but instanly got an exception "failed to invoke ShowJobPlanIFrame"
Is there any way to handle this task?
"Out of browser" mode doesn't fit.
Thanks.
===========================================================================
Update:
It worked out using IFrame overlay.
Button click invokes the following code in C#:
var scriptObject = (ScriptObject)HtmlPage.Window.GetProperty("ShowJobPlanIFrame");
scriptObject.InvokeSelf(url);
Where "ShowJobPlanIFrame" is as defined at:
http://weblogs.asp.net/dwahlin/archive/2010/05/10/integrating-html-into-silverlight-applications.aspx
This allowed me to pass data into XML editor and then get it back.
An error with JavaScript function invocation I told above, was my fault in JavaScript code itself.
A very similar scenario: https://stackoverflow.com/a/7919065/384316
Try using an iframe overlay, then you can load any HTML-like content.
There is an excellent explanation of how to do this here:
http://www.wintellect.com/cs/blogs/jlikness/archive/2010/09/19/hosting-html-in-silverlight-not-out-of-browser.aspx
It worked out using IFrame overlay.
Button click invokes the following code in C#:
var scriptObject = (ScriptObject)HtmlPage.Window.GetProperty("ShowJobPlanIFrame");
scriptObject.InvokeSelf(url);
Where "ShowJobPlanIFrame" is as defined at:
http://weblogs.asp.net/dwahlin/archive/2010/05/10/integrating-html-into-silverlight-applications.aspx
This allowed me to pass data into XML editor and then get it back.
An error with JavaScript function invocation I told above, was my fault in JavaScript code itself.
Did you try NavigationFramework of Silverlight? It's capability may support your needs in a more simple way than using multiple browser pages.
I have a problem receiving and opening a picture via AJAX.
If I call the following page:
http://127.0.0.1:8889/ex?sql=SELECT+Image+FROM+Persons+WHERE+Number+Like+%27%2501%27
a picture is displayed from a blob field in IE8.
Now I would like to open this into a div after someone pressed a key (using AJAX)?
Trying to use xhr.responseText does not work (I get an error. Using it on a text response works). So it seems that my problem is to grab the result from the ajax request.
How can I do this?
Some code and the error message:
var picReturn = xhr.responseText;
=> Could not continue due to the following error: c00ce514
You have three options:
Place the resultant data in an iframe. Not very practical.
Take the result and place it in am image source as a data:uri. Not supported in older browsers and limited to 32/64Kb depending on the browser.
Skip the AJAX and write a web service and use that as your url. This is the best option.
You don't say what language you're using server-side but you essentially want to open a web response, set the header to "image/jpeg" and return your stream.