I'm trying to scrape a country from linkedin profile that's displayed on google results.
When I try to get info that's on this search results snippet through the following code:
All I get is the 'description' of this search result and not the grey part;
var country=JSON.parse(response).items[0].snippet
Related
I want to import data from a table from the following page:
https://basketballmonster.com/playerrankings.aspx
When I do so, with all players filter selected, only the top players are imported to my googlesheet. Can someone help me achieve this? Appreciation in advance.
I attached the googlesheet below for your review:
https://docs.google.com/spreadsheets/d/1uvhNp6gBnnEvs8CBb4K7onccew_doFp96wmFEsYyLBk/edit?usp=sharing
Google Sheets can't know what your browser has, so it doesn't know whether you selected which filter. You have to get the same html for Sheets that you have displayed in your browser, which means adding the filter in your query.
Since it looks like the controls aren't passed as parameters in a GET request, sadly it's not as simple as appending
?PlayerFilterControl=TopPlayers to the url.
You have to POST it as a payload with a post request like
{ 'PlayerFilterControl': 'AllPlayers'}.
Sadly Google Sheets IMPORTHTML() doesn't support post request yet, so you'll have to get into apps scripts, request and xml parsing.
I suggest you check out these:
https://developers.google.com/apps-script/reference/url-fetch/url-fetch-app
https://developers.google.com/apps-script/reference/xml-service
I am relatively new to programming and I am making a HTML website where I want to show weather data. I have got an API from openweathermap.org.
When I type the API in google I get data in JSON code.
{"coord":{"lon":5.46,"lat":51.35},
"weather":[{"id":804,"main":"Clouds","description":"overcast clouds","icon":"04d"}],
"base":"stations","main":
{"temp":5.72,"feels_like":1.32,"temp_min":5.56,"temp_max":6.11,
"pressure":998,"humidity":70},"visibility":10000,"wind":{"speed":3.6,"deg":160},
"clouds":{"all":90},"dt":1607178963,"sys":
{"type":1,"id":1527,"country":"NL","sunrise":1607153201,"sunset":1607182290},
"timezone":3600,"id":2745860,"name":"Valkenswaard","cod":200}
The problem is that I don't know how to get the data into my HTML website.
I would like to know how to get JSON code into a HTML website.
I'm trying to import some energy data into a Google Sheet, but can't seem to get the value to appear since it is just text in the HTML line, and shows up as --,--- in the output. I have tried adding /text() to the XPath but that is still not working either.
Website with data to import:
http://www.caiso.com/TodaysOutlook/Pages/default.aspx
Formula in Google Sheets:
=IMPORTXML("http://www.caiso.com/TodaysOutlook/Pages/default.aspx","//div[#class='overview-large-number']")
Also tried the follwoing Xpath with the same --,--- being displayed.
/html/body/div[2]/div[3]/div[4]/div/div/div[2]/div/div[1]/div[1]/div[1]
/html/body/div[2]/div[3]/div[4]/div/div/div[2]/div/div[1]/div[1]/div[1]/text()
Screen Shot of Webpage:
Caiso Webage with Data
HTML Element with Data:
<div class="overview-large-number">41,946 <span>MW</span></div>
Screen Shote of Google Sheets current Output:
Google Sheets Screenshot
Thank you in advanced for any insight.
The data is populated after the page loads, so you can't use ImportXML(). The data is requested via HTTP Get on the URL http://www.caiso.com/outlook/SP/stats.txt and is returned in JSON format.
Get the ImportJSON script from here and add it to your sheet (Tools > Script Editor).
This should be enough to get you started:
=Transpose(ArrayFormula(HLOOKUP(
{"AvailableCapacity","CurrentSystemDemand","todayForecastPeakDemand","histDemand","TodaysPeakDemand","tomorrowsForecastPeakDemand"},
ImportJSON("http://www.caiso.com/outlook/SP/stats.txt"),2,0)))
Note that the Historical Peak value is already a string, so you may have to do some extra work to get the number out.
I'm trying to build a custom image search using Google's CSE and I'm facing some difficulties getting the URL from the result box.
I have already created a new search engine and implemented the code on my web page.
The results are showing properly and the search engine is working as intended.
Now what I would like to get is:
Avoid opening the picture on another window
Click on the image and get the real URL of the picture (not encrypted)
Display selected url on alert window
What I found so far exploring the source code is that the result image items URL links are encrypted and not showing the real URL of the image.
The following code is the result of searching the same dog using my custom search engine with Google's API and Google image search web page.
The img result using my custom search engine
<img src="https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQmjyo4iFP_SQ-a2u7piFW8hZ_J4CEtDMF6_dE8Ev-RvqcZlA93DIo6uQo" title="How dogs contribute to your health and happiness" class="gs-image gs-image-scalable" style="width: 126px; height: 84px;">
The img result using Google's web image search
<img class="irc_mi" src="https://cdn1.medicalnewstoday.com/content/images/articles/322/322868/golden-retriever-puppy.jpg" onload="typeof google==='object'&&google.aft&&google.aft(this)" alt="Image result for dogs" width="529" height="353" style="margin-top: 0px;" data-iml="1552180024897">
Note that src parameter in one is encrypted and not in second one. If I do the same search on Google image I can get the real URL from the src.
So my question is: How to get the real image URL using Google's CSE? Is there any way I can get the original URL?
I have gone through this the documentation https://developers.google.com/custom-search/v1/cse/list but is not clear to me how to implement the Optional query parameters. Maybe with "linkSite" query parameter?
Best regards and have a nice day!
I'm using the Twitter URL: http://search.twitter.com/search.json?q=%23test in order to return a JSON list of tweets with the hash tag #Test.
It returns 15 results and says page 1. How do I load the next page of 15 tweets?
*I tried adding &page=2 to the search URL, but no luck.
The JSON response of the URI you posted includes the query parameters to use to get the next page. It's in the "next_page" element. Currently, the value is "?page=2&max_id=80076124637499392&q=%23test". So, the complete URI for the second page is http://search.twitter.com/search.json?page=2&max_id=80076124637499392&q=%23test
The Twitter search API documentation unfortunately does not describe how to use this parameter, though it is briefly described in the Twitter API Wiki at http://apiwiki.twitter.com/w/page/22554664/Return-Values.