working with advavced json data anjular js - json

i am new to anjular js , i know how to work with basic json data with anjular js.
i have nauseated json data
[
{
"activity_user": "asd#gmail.com",
"home_id": "1",
"recent_connect_address": "South Hill Road, Callington, Cornwall ",
"recent_connect_postcode": "WA3 1PQ",
"propertyimg": "a.jpg",
"datemovein": "2014-12-04 00:00:00",
"datemoveout": "2016-12-29 00:00:00",
"list": "[{ comment:\"The $190 Bonavita 1900TS made better coffee than the other machines, according to our 10-person tasting panel. \", date:\"2014-12-01 00:00:00\"},{ comment:\"The $190 Bonavita 1900TS made better coffee than the other machines, according to our 10-person tasting panel. \", date:\"2014-12-01 00:00:00\"}]"
},
{
"activity_user": "asd525#gmail.com",
"home_id": "2",
"recent_connect_address": "548 Newton Road, Lowton, Warrington ",
"recent_connect_postcode": "PL17 7LH",
"propertyimg": "a.jpg",
"datemovein": "2014-12-01 00:00:00",
"datemoveout": "2014-12-31 00:00:00",
"list": "[{ comment:\"We considered 80 Champagne glasses before testing 10 glasses for 12 hours, and we found that the Schott Zwiesel 1872 Enoteca is best for most people. It’s taller, lighter, and thinner than any glass we tried, with tiny etching to keep Champagne carbonated longer. The tulip shape allows more aromas to reach your nose while still maintaining an elegant profile.\", date:\"2014-12-31 00:00:00\"}]"
}
]
now i want to print it out following format
<div class="row" ng-controller="ListCtrl">
<div ng-repeat="property in timeline" >
<div>{{property.activity_user}}</div>
<div class="comments">
<div>{{property.list.}}</div>
</div>
</div>
here is my controller but it not work
function ListCtrl($scope, $http) {
$http({method: 'GET', url: 'my.json'}).success(function(data) {
$scope.timeline = data;
});
};
i refer Accesing nested JSON with AngularJS but i didn't understand it

There are several problems in your code right now:
DEMO
The interpolated value inside your ng-repeat for the property.list has a dot in it:
change
<div>{{property.list.}}</div>
to
<div>{{property.list}}</div>
You are missing a div element in your html for closing the top level div.
You are declaring your controllers in a global manner, this is already deprecated and no longer recommended as of AngularJS 1.3. See documentation Arguments Section.
Instead of declaring it like this:
function ListCtrl() {}
You can do this instead:
angular.module('yourApp', [])
.controller('ListCtrl', function() {});
The list property is a string, not an array of objects representing comments.
I suggest you change its structure to something like this:
"list": [
{
"comment": "The $190 Bonavita 1900TS made better coffee than the other machines, according to our 10-person tasting panel. ",
"date":"2014-12-01 00:00:00"
},
{
"comment": "The $190 Bonavita 1900TS made better coffee than the other machines, according to our 10-person tasting panel. ",
"date":"2014-12-01 00:00:00"
}]
Javascript
angular.module('demo', [])
.controller('ListCtrl', function($scope, $http) {
$http({method: 'GET', url: 'my.json'}).success(function(data) {
$scope.timeline = data;
});
});
HTML
<div class="row" ng-controller="ListCtrl">
<div ng-repeat="property in timeline">
<div>{{property.activity_user}}</div>
<div class="comments">
<div ng-repeat="item in property.list">
<div>{{item.comment}}</div>
<em>-- {{item.date}}</em>
</div>
</div>
</div>
</div>
my.json
[
{
"activity_user": "asd#gmail.com",
"home_id": "1",
"recent_connect_address": "South Hill Road, Callington, Cornwall ",
"recent_connect_postcode": "WA3 1PQ",
"propertyimg": "a.jpg",
"datemovein": "2014-12-04 00:00:00",
"datemoveout": "2016-12-29 00:00:00",
"list": [
{
"comment": "The $190 Bonavita 1900TS made better coffee than the other machines, according to our 10-person tasting panel. ",
"date":"2014-12-01 00:00:00"
},
{
"comment": "The $190 Bonavita 1900TS made better coffee than the other machines, according to our 10-person tasting panel. ",
"date":"2014-12-01 00:00:00"
}
]
},
{
"activity_user": "asd525#gmail.com",
"home_id": "2",
"recent_connect_address": "548 Newton Road, Lowton, Warrington ",
"recent_connect_postcode": "PL17 7LH",
"propertyimg": "a.jpg",
"datemovein": "2014-12-01 00:00:00",
"datemoveout": "2014-12-31 00:00:00",
"list": [
{
"comment": "We considered 80 Champagne glasses before testing 10 glasses for 12 hours, and we found that the Schott Zwiesel 1872 Enoteca is best for most people. It’s taller, lighter, and thinner than any glass we tried, with tiny etching to keep Champagne carbonated longer. The tulip shape allows more aromas to reach your nose while still maintaining an elegant profile",
"date":"2014-12-31 00:00:0"
}
]
}
]

If you re using ( $scope.timeline ), that means you must use in html :
{{timeline[0].activity_user}} // print: asd#gmail.com

Related

How to properly display JSON data in Vue.js?

I am recently new to Vue and I am trying to show JSON data in the page I am currently working on.
I've managed to have the whole JSON show and now I am trying to properly show it without brackets or the object keys on it and where each data occupies one line on the page.
Would anyone please assist me on this?
Here is the code:
<template>
<div class="CityGuidelines">
<v-card class="guidelines">
<h2>test</h2>
<p>This is a test for the new guidelines page card</p>
{{ info }}
</v-card>
</div>
</template>
<script>
import Info from '#/data/info.json'
export default {
name: 'CityGuidelines',
data() {
return {
info: Info.
}
}
}
</script>
Part of the JSON data
{
"San Francisco County": {
"info": [
{
"header": "County Guidelines",
"body": [
{
"h4": "Shelter in Place",
"content": [
{
"p2": [
{
"p": "Residents should remain at home when possible as it is the safest choice. If you do leave your home, keep the following guidelines in mind."
}
],
"bullets": [
{
"bullet": "Maintain 6ft distance between yourself and people who are not in your hosehold unit."
},
{
"bullet": "Wear a mask, scarf, bandana or other facial covering."
},
{
"bullet": "Limit face to face interactions with those vulnerable to the coronavirus such as those over 60 or those with underlying health conditions."
},
{
"bullet": "Wash hands frequently with soap and water for at least 20 seconds, or use hand sanitizer."
},
{
"bullet": "Cover coughs and sneezes with a tissue or fabric or, if not possible, use the sleeve of your shirt or elbow. Do not cough or sneeze into your hand."
},
{
"bullet": "Use antibactiral wipes to clean any surface that you have to touch."
}
],
"link": "https://sf.gov/stay-home-except-essential-needs"
}
]
},

How to match JSON-LD markup with REGEX?

I’m attempting to match an entire json-ld entry, regardless of specific markup, line breaks, etc.
Why doesn’t something as simple as this work:
\<script type\=\"application\/ld\+json\"\>(.*?)\<\/script\>
<script type="application/ld+json">
{
"#context": "http://schema.org",
"#type": "Recipe",
"author": "John Smith",
"cookTime": "PT1H",
"datePublished": "2009-05-08",
"description": "This classic banana bread recipe comes from my mom -- the walnuts add a nice texture and flavor to the banana bread.",
"image": "bananabread.jpg",
"recipeIngredient": [
"3 or 4 ripe bananas, smashed",
"1 egg",
"3/4 cup of sugar"
],
"interactionStatistic": {
"#type": "InteractionCounter",
"interactionType": "http://schema.org/Comment",
"userInteractionCount": "140"
},
"name": "Mom's World Famous Banana Bread",
"nutrition": {
"#type": "NutritionInformation",
"calories": "240 calories",
"fatContent": "9 grams fat"
},
"prepTime": "PT15M",
"recipeInstructions": "Preheat the oven to 350 degrees. Mix in the ingredients in a bowl. Add the flour last. Pour the mixture into a loaf pan and bake for one hour.",
"recipeYield": "1 loaf",
"suitableForDiet": "http://schema.org/LowFatDiet"
}
</script>
I expect the output to be everything inside the tag.
Here, we might want to bound our expression with an open json/ld tag as a start boundary, then collect all chars and newlines, and finally add a right boundary with closing script tag, maybe similar to:
(<script type="application\/ld\+json">)([\s\S]*)(<\/script>)
or
^(<script type="application\/ld\+json">)([\w\W]*)(<\/script>)$
DEMO
However, maybe here it would not be the best idea to user regular expressions and there should be so many methods that would do so much easier.
Test
const regex = /^(<script type="application\/ld\+json">)([\w\W]*)(<\/script>)$/gm;
const str = `<script type="application/ld+json">
{
"#context": "http://schema.org",
"#type": "Recipe",
"author": "John Smith",
"cookTime": "PT1H",
"datePublished": "2009-05-08",
"description": "This classic banana bread recipe comes from my mom -- the walnuts add a nice texture and flavor to the banana bread.",
"image": "bananabread.jpg",
"recipeIngredient": [
"3 or 4 ripe bananas, smashed",
"1 egg",
"3/4 cup of sugar"
],
"interactionStatistic": {
"#type": "InteractionCounter",
"interactionType": "http://schema.org/Comment",
"userInteractionCount": "140"
},
"name": "Mom's World Famous Banana Bread",
"nutrition": {
"#type": "NutritionInformation",
"calories": "240 calories",
"fatContent": "9 grams fat"
},
"prepTime": "PT15M",
"recipeInstructions": "Preheat the oven to 350 degrees. Mix in the ingredients in a bowl. Add the flour last. Pour the mixture into a loaf pan and bake for one hour.",
"recipeYield": "1 loaf",
"suitableForDiet": "http://schema.org/LowFatDiet"
}
</script>`;
const subst = `$2`;
// The substituted value will be contained in the result variable
const result = str.replace(regex, subst);
console.log('Substitution result: ', result);
DEMO
RegEx
If this expression wasn't desired, it can be modified or changed in regex101.com.
RegEx Circuit
jex.im visualizes regular expressions:

Get request to Google Search

I'm trying to get HTML with search results from Google. With sending GET request for example to:
https://www.google.ru/?q=1111
But if in browser all is ok, when I'm trying to use it with curl or to get source with "View source" in Google, there is only some Javascript code, no search result. Is that some type of protection? What can I do?
You now have to use the Google Search API to make your GET requests.
All other methods have been blocked.
The page from your question is the Google Search page with the input field.
The search results page is this one:
https://www.google.ru/search?q=1111
Rotate proxies and user agents, and delay similar requests to get the HTML from Google Search results pages with fewer amount of bans.
Or use SerpApi to access HTML and the extracted data from it. It has a free trial.
curl -s 'https://serpapi.com/search?q=coffee'
Output
{
// Omitted
"organic_results": [
{
"position": 1,
"title": "Coffee - Wikipedia",
"link": "https://en.wikipedia.org/wiki/Coffee",
"displayed_link": "en.wikipedia.org › wiki › Coffee",
"snippet": "Coffee is a brewed drink prepared from roasted coffee beans, the seeds of berries from certain Coffea species. When coffee berries turn from green to bright red ...",
"sitelinks": {
"expanded": [
{
"title": "History",
"link": "https://en.wikipedia.org/wiki/History_of_coffee",
"snippet": "The history of coffee dates back to the 15th century, and possibly ..."
},
{
"title": "International Coffee Day",
"link": "https://en.wikipedia.org/wiki/International_Coffee_Day",
"snippet": "International Coffee Day (1 October) is an occasion that is ..."
},
{
"title": "List of coffee drinks",
"link": "https://en.wikipedia.org/wiki/List_of_coffee_drinks",
"snippet": "Milk coffee - Nitro cold brew coffee - List of coffee dishes - ..."
},
{
"title": "Portal:Coffee",
"link": "https://en.wikipedia.org/wiki/Portal:Coffee",
"snippet": "Coffee is a brewed drink prepared from roasted coffee beans, the ..."
},
{
"title": "Coffee bean",
"link": "https://en.wikipedia.org/wiki/Coffee_bean",
"snippet": "A coffee bean is a seed of the Coffea plant and the source for ..."
},
{
"title": "Geisha",
"link": "https://en.wikipedia.org/wiki/Geisha_(coffee)",
"snippet": "Geisha coffee, sometimes referred to as Gesha coffee, is a type of ..."
}
],
"list": [
{
"date": "Color‎: ‎Black, dark brown, light brown, beige"
}
]
},
"rich_snippet": {
"bottom": {
"detected_extensions": {
"introduced_th_century": 15
},
"extensions": [
"Introduced‎: ‎15th century",
"Color‎: ‎Black, dark brown, light brown, beige"
]
}
},
"cached_page_link": "https://webcache.googleusercontent.com/search?q=cache:U6oJMnF-eeUJ:https://en.wikipedia.org/wiki/Coffee+&cd=2&hl=sv&ct=clnk&gl=se",
"related_pages_link": "https://www.google.se/search?gl=se&hl=sv&q=related:https://en.wikipedia.org/wiki/Coffee+coffee&sa=X&ved=2ahUKEwjJ9p2p_KXuAhVlRN8KHf22D8wQHzABegQIAhAJ"
}
},
// ...
}
Disclaimer: I work at SerpApi.
To add a bit more sauce to the answers as they are not correct and do not even respond to your problem.
First of all, it's perfectly legal to scrape Google as long as you do not harm their service through it (DoS-like).
Also the methods have not been blocked, it's just not that simple.
The speed depends on your methods, it does not have to be very slow..
You can scrape ten thousands of keyword pages in a minute if needed.
You will find a better answer to the topic here: Is it ok to scrape data from Google results?
Your problem with curl comes indeed from protection, Google does not allow automated access and it has a very sophisticated set of detection algorithms.
They go from simple user agent checks (that's what stopped you directly) up to artificial intelligence that tries to detect unusual queries or related queries.
You can load it in the browser and then scrape results via Javascript.
Or you can use Google API, but seems that it requires payment if you will request it more then 100 times per day.

Repeating over nested Objects with Angular JS

I've been playing with the google feed API for a podcasts I run and wanted to include a simple ng-repeat to display the title and link URL to the MP3. However the JSON google provides is nested in several different Objects and Arrays. For instance, my JSON feed looks like this:
{
"responseData": {
"feed": {
"feedUrl": "http://feeds.feedburner.com/stillgotgame",
"title": "2old2play presents Still Got Game",
"link": "http://www.2old2play.com/",
"author": "",
"description": "Still Got Game focuses on the gaming industry from the perspective of adult gamers. We look at news, reviews, and inside information in the world of video games. Each episode touches on the community, the industry, and the games that keep us coming back.",
"type": "rss20",
"entries": [
{
"mediaGroups": [
{
"contents": [
{
"url": "http://traffic.libsyn.com/dsmooth/Still_Got_Game_Episode_33__Coast_to_Coast.mp3",
"fileSize": "35346436",
"type": "audio/mpeg"
}
]
}
],
"title": "Still Got Game Ep. 33: Coast to Coast",
"link": "http://2old2play.com/media/still-got-game-ep-33-coast-coast/",
"author": "podcast#2old2play.com",
"publishedDate": "Tue, 06 May 2014 22:05:01 -0700",
"contentSnippet": "DSmooth finally has his Rocket Bro back. After a multi-week hiatus for Doodirock's move to the West Coast, they boys were back ...",
"content": "DSmooth finally has his Rocket Bro back. After a multi-week hiatus for Doodirock's move to the West Coast, they boys were back in force this week. The duo talk gaming news and the new releases, cover a bunch of viewer feedback, and talk a bit about what may be the worst moving company ever. They'll have you LMFAOing! You can always call the boys at (773) 527-2961 and weigh in yourself, or tune in live Monday nights at 9:00 EDT at http://twitch.tv/2old2play ...",
"categories": [
"Audio"
]
}
]
}
},
"responseDetails": null,
"responseStatus": 200
}
As you can see, in order to get to the items URL to the MP3 I have to go through entries, mediaGroups, and Contents before I even reach the Array I need! I start off inside the entries with this factory I've created:
.factory('audioFEED', function($resource){
return $resource('http://ajax.googleapis.com/ajax/services/feed/load?v=1.0&num=100&q=http://feeds.feedburner.com/stillgotgame',{},
{
query:{
method:'JSONP',
params: {callback: 'JSON_CALLBACK'},
isArray:false,
headers:{
'Access-Control-Allow-Origin': '*'
}
},
});
});
Thats easy enough with just setting up the data on the controller here:
'use strict';
angular.module('twitchappApp')
.controller('audioCtrl', function($scope, audioFEED) {
audioFEED.query(function(data){
$scope.audios = data.responseData.feed.entries;
console.log($scope.audios);
});
});
However, In order to get to that data I'm having to set up multiple ng-repeats with on inside of the next. I would really like to find a better way to handle this data within the controller and access the URL inside one ng-repeat. It seems this way is adding more over head and probably not the best over all method. Is there a best practice for this? My current end result looks like this:
<h1>Audio</h1>
<div ng-repeat="audio in audios">
<h3>{{ audio.title }}</h3>
<p>{{audio.contentSnippet}}</p>
<div ng-repeat="play in audio.mediaGroups">
<div ng-repeat="playurl in play.contents">
PLAY
</div>
</div>
</div>
Yuk...
Check out this JSFiddle. Uses underscore to flatten your data down to an easier to work with array. http://jsfiddle.net/ahchurch/sKeY9/3/
Template
<div ng-controller="MyCtrl">
<div ng-repeat="playurl in contents">
PLAY
</div>
</div>
JavaScript
var myApp = angular.module('myApp',[]);
//myApp.directive('myDirective', function() {});
//myApp.factory('myService', function() {});
function MyCtrl($scope) {
var responseData = {
"responseData": {
"feed": {
"feedUrl": "http://feeds.feedburner.com/stillgotgame",
"title": "2old2play presents Still Got Game",
"link": "http://www.2old2play.com/",
"author": "",
"description": "Still Got Game focuses on the gaming industry from the perspective of adult gamers. We look at news, reviews, and inside information in the world of video games. Each episode touches on the community, the industry, and the games that keep us coming back.",
"type": "rss20",
"entries": [
{
"mediaGroups": [
{
"contents": [
{
"url": "http://traffic.libsyn.com/dsmooth/Still_Got_Game_Episode_33__Coast_to_Coast.mp3",
"fileSize": "35346436",
"type": "audio/mpeg"
}
]
}
],
"title": "Still Got Game Ep. 33: Coast to Coast",
"link": "http://2old2play.com/media/still-got-game-ep-33-coast-coast/",
"author": "podcast#2old2play.com",
"publishedDate": "Tue, 06 May 2014 22:05:01 -0700",
"contentSnippet": "DSmooth finally has his Rocket Bro back. After a multi-week hiatus for Doodirock's move to the West Coast, they boys were back ...",
"content": "DSmooth finally has his Rocket Bro back. After a multi-week hiatus for Doodirock's move to the West Coast, they boys were back in force this week. The duo talk gaming news and the new releases, cover a bunch of viewer feedback, and talk a bit about what may be the worst moving company ever. They'll have you LMFAOing! You can always call the boys at (773) 527-2961 and weigh in yourself, or tune in live Monday nights at 9:00 EDT at http://twitch.tv/2old2play ...",
"categories": [
"Audio"
]
}
]
}
},
"responseDetails": null,
"responseStatus": 200
};
//Underscore:
$scope.contents = _.flatten(_.map(responseData.responseData.feed.entries, function(entry){
return _.map(entry.mediaGroups, function(mediaGroup){
return mediaGroup.contents;
});
}));
$scope.name = 'Superhero';
}

css selector using Cheerio exclusion of double class

I have a rather "weird" scenario (web scraping). I want to select class=g only but not those with class="g g" (double class g). How to do that in jQuery?
If I use $('.g'), it will select both .g and .g .g
UPDATE 1:
If you don't think .g .g is valid, do View Source in Google search results ;)
Even if I don't think g g is valid (Let me check it)..and therefore $(".g:not(.g.g)") wouldn't work, you could do something like:
var myWeirdElements = $('.g').map(function(){return this.className.indexOf('g g') && this ;});
// or, as mentioned on the comment, using Array.filter method
demo => http://jsfiddle.net/GeAaA/
Edit: about the 2nd comment, you just have to count how many g (simple regex)
Considering the following scenario where you have two Div tag with above classes
<div class="g"></div>
<div class="g g"></div>
Write following to get only those div which have class"g"
alert($("div[class='g']").length);
As an alternative solution to other answers, you can use SerpApi's Google Search Engine Results API. This API makes it possible to forget about figuring out how to extract something from the page, bypass blocks from search engines, and maintain the code (parser) over time.
All you need is to iterate over the structured JSON and get the data you want. Check out the playground.
Usage example (also, check it in the online IDE):
const SerpApi = require('google-search-results-nodejs');
const search = new SerpApi.GoogleSearch(process.env.API_KEY); //your api key from serpapi.com
const params = {
engine: "google", // search engine
q: "Coffee", // search query
google_domain: "google.com", // google domain of the search
gl: "us", // parameter defines the country to use for the Google search
hl: "en", // Parameter defines the language to use for the Google search
};
const callback = function(data) {
console.log(data);
};
// Show result as JSON
search.json(params, callback);
Output:
{
"recipes_results":[
{
"title":"Coffee recipes",
"link":"https://www.bbcgoodfood.com/recipes/collection/coffee-recipes",
"source":"BBC Good Food",
"ingredients":[
"Instant coffee"
],
"thumbnail":"https://serpapi.com/searches/62bda4a2f0adfb0a9364cfb4/images/b50a38f136c7abe67e61eef25884f35798c0551f1d43ace6237f6830200fa235.jpeg"
},
{
"title":"20 Great Coffee Drinks From Around the World",
"link":"https://insanelygoodrecipes.com/coffee-recipes/",
"source":"Insanely Good Recipes",
"rating":4.5,
"reviews":8,
"ingredients":[
"Turkish coffee",
"vietnamese coffee",
"white chocolate mocha",
"cold brew coffee",
"frappuccino"
],
"thumbnail":"https://serpapi.com/searches/62bda4a2f0adfb0a9364cfb4/images/b50a38f136c7abe67e61eef25884f357aa875ca61585258439c617a0846d590a.jpeg"
},
{
"title":"Dalgona Coffee",
"link":"https://www.delish.com/cooking/recipe-ideas/a32072159/dalgona-coffee-recipe/",
"source":"Delish.com",
"rating":3.8,
"reviews":42,
"total_time":"10 min",
"ingredients":[
"Instant coffee",
"ice",
"milk"
],
"thumbnail":"https://serpapi.com/searches/62bda4a2f0adfb0a9364cfb4/images/b50a38f136c7abe67e61eef25884f357b2162e9f1c396805c73a0850d499fd01.jpeg"
}
],
"local_map":{
"link":"https://www.google.com/search?gl=us&hl=en&q=Coffee&npsic=0&rflfq=1&rldoc=1&rllag=37769342,-122389993,242&tbm=lcl&sa=X&ved=2ahUKEwiFz_LLo9X4AhUwGlkFHeVCB84QtgN6BAgXEAE",
"image":"https://serpapi.com/searches/62bda4a2f0adfb0a9364cfb4/images/57f06389eb0c57c24320522b45952370.png",
"gps_coordinates":{
"latitude":37.769342,
"longitude":-122.389993,
"altitude":242
}
},
"local_results":{
"more_locations_link":"https://www.google.com/search?gl=us&hl=en&tbs=lf:1,lf_ui:9&tbm=lcl&q=Coffee&rflfq=1&num=10&sa=X&ved=2ahUKEwiFz_LLo9X4AhUwGlkFHeVCB84QjGp6BAgWEAI",
"places":[
{
"position":1,
"title":"Philz Coffee",
"place_id":"14850561974798048043",
"lsig":"AB86z5V2CpkHAoUti4UAfsYs_iF5",
"place_id_search":"https://serpapi.com/search.json?device=desktop&engine=google&gl=us&google_domain=google.com&hl=en&lsig=AB86z5V2CpkHAoUti4UAfsYs_iF5&ludocid=14850561974798048043&q=Coffee&tbm=lcl",
"description":"Dine-in·Takeout·Delivery",
"rating":4.5,
"type":"Philz Coffee",
"hours":"San Francisco, CA · In Chase Center",
"address":"4.5(21) · $$ · Coffee shop",
"thumbnail":"https://serpapi.com/searches/62bda4a2f0adfb0a9364cfb4/images/db978ef22623ebbaba5a259d7610a82f1e67d73393884509c7d902e95abeda94a9e2ac25fedcf5ad.jpeg"
}, ... other results
]
},
"organic_results":[
{
"position":1,
"title":"Coffee - Wikipedia",
"link":"https://en.wikipedia.org/wiki/Coffee",
"displayed_link":"https://en.wikipedia.org › wiki › Coffee",
"thumbnail":"https://serpapi.com/searches/62bda4a2f0adfb0a9364cfb4/images/0293bd7aa87e25161235b6fe723a6c2dae65c3f9f164074026df984d8f3e5820.jpeg",
"snippet":"Coffee is a brewed drink prepared from roasted coffee beans, the seeds of berries from certain flowering plants in the Coffea genus. From the coffee fruit, ...",
"snippet_highlighted_words":[
"Coffee",
"coffee",
"coffee"
],
"sitelinks":{
"inline":[
{
"title":"Coffee bean",
"link":"https://en.wikipedia.org/wiki/Coffee_bean"
},
{
"title":"History",
"link":"https://en.wikipedia.org/wiki/History_of_coffee"
},
{
"title":"Coffee preparation",
"link":"https://en.wikipedia.org/wiki/Coffee_preparation"
},
{
"title":"Coffee production",
"link":"https://en.wikipedia.org/wiki/Coffee_production"
}
]
},
"rich_snippet":{
"bottom":{
"extensions":[
"Region of origin: Horn of Africa and ‎South Ara...‎",
"Color: Black, dark brown, light brown, beige",
"Introduced: 15th century"
],
"detected_extensions":{
"introduced_th_century":15
}
}
},
"about_this_result":{
"source":{
"description":"Wikipedia is a multilingual free online encyclopedia written and maintained by a community of volunteers through open collaboration and a wiki-based editing system. Individual contributors, also called editors, are known as Wikipedians. Wikipedia is the largest and most-read reference work in history.",
"icon":"https://serpapi.com/searches/62bda4a2f0adfb0a9364cfb4/images/0293bd7aa87e25161235b6fe723a6c2d8021773b9a274d74b2c9d60da12b643d997acd2685004b3a989713a5bd03429c.png"
},
"keywords":[
"coffee"
],
"languages":[
"English"
],
"regions":[
"the United States"
]
},
"about_page_link":"https://www.google.com/search?q=About+https://en.wikipedia.org/wiki/Coffee&tbm=ilp&ilps=ADNMCi0tVhSB-fGHOJYgrIxB0xlXYrPGPA",
"cached_page_link":"https://webcache.googleusercontent.com/search?q=cache:U6oJMnF-eeUJ:https://en.wikipedia.org/wiki/Coffee+&cd=13&hl=en&ct=clnk&gl=us",
"related_pages_link":"https://www.google.com/search?gl=us&hl=en&q=related:https://en.wikipedia.org/wiki/Coffee+Coffee"
}, ... other results
],
... and other search results
}
Disclaimer, I work for SerpApiAs an alternative solution to other answers, you can use SerpApi's Google Search Engine Results API. This API makes it possible to forget about figuring out how to extract something from the page, bypass blocks from search engines, and maintain the code (parser) over time.
All you need is to iterate over the structured JSON and get the data you want. Check out the playground.
Usage example (also, check it in the online IDE):
const SerpApi = require('google-search-results-nodejs');
const search = new SerpApi.GoogleSearch(process.env.API_KEY); //your api key from serpapi.com
const params = {
engine: "google", // search engine
q: "Coffee", // search query
google_domain: "google.com", // google domain of the search
gl: "us", // parameter defines the country to use for the Google search
hl: "en", // Parameter defines the language to use for the Google search
};
const callback = function(data) {
console.log(data);
};
// Show result as JSON
search.json(params, callback);
Output:
{
"recipes_results":[
{
"title":"Coffee recipes",
"link":"https://www.bbcgoodfood.com/recipes/collection/coffee-recipes",
"source":"BBC Good Food",
"ingredients":[
"Instant coffee"
],
"thumbnail":"https://serpapi.com/searches/62bda4a2f0adfb0a9364cfb4/images/b50a38f136c7abe67e61eef25884f35798c0551f1d43ace6237f6830200fa235.jpeg"
},
{
"title":"20 Great Coffee Drinks From Around the World",
"link":"https://insanelygoodrecipes.com/coffee-recipes/",
"source":"Insanely Good Recipes",
"rating":4.5,
"reviews":8,
"ingredients":[
"Turkish coffee",
"vietnamese coffee",
"white chocolate mocha",
"cold brew coffee",
"frappuccino"
],
"thumbnail":"https://serpapi.com/searches/62bda4a2f0adfb0a9364cfb4/images/b50a38f136c7abe67e61eef25884f357aa875ca61585258439c617a0846d590a.jpeg"
},
{
"title":"Dalgona Coffee",
"link":"https://www.delish.com/cooking/recipe-ideas/a32072159/dalgona-coffee-recipe/",
"source":"Delish.com",
"rating":3.8,
"reviews":42,
"total_time":"10 min",
"ingredients":[
"Instant coffee",
"ice",
"milk"
],
"thumbnail":"https://serpapi.com/searches/62bda4a2f0adfb0a9364cfb4/images/b50a38f136c7abe67e61eef25884f357b2162e9f1c396805c73a0850d499fd01.jpeg"
}
],
"local_map":{
"link":"https://www.google.com/search?gl=us&hl=en&q=Coffee&npsic=0&rflfq=1&rldoc=1&rllag=37769342,-122389993,242&tbm=lcl&sa=X&ved=2ahUKEwiFz_LLo9X4AhUwGlkFHeVCB84QtgN6BAgXEAE",
"image":"https://serpapi.com/searches/62bda4a2f0adfb0a9364cfb4/images/57f06389eb0c57c24320522b45952370.png",
"gps_coordinates":{
"latitude":37.769342,
"longitude":-122.389993,
"altitude":242
}
},
"local_results":{
"more_locations_link":"https://www.google.com/search?gl=us&hl=en&tbs=lf:1,lf_ui:9&tbm=lcl&q=Coffee&rflfq=1&num=10&sa=X&ved=2ahUKEwiFz_LLo9X4AhUwGlkFHeVCB84QjGp6BAgWEAI",
"places":[
{
"position":1,
"title":"Philz Coffee",
"place_id":"14850561974798048043",
"lsig":"AB86z5V2CpkHAoUti4UAfsYs_iF5",
"place_id_search":"https://serpapi.com/search.json?device=desktop&engine=google&gl=us&google_domain=google.com&hl=en&lsig=AB86z5V2CpkHAoUti4UAfsYs_iF5&ludocid=14850561974798048043&q=Coffee&tbm=lcl",
"description":"Dine-in·Takeout·Delivery",
"rating":4.5,
"type":"Philz Coffee",
"hours":"San Francisco, CA · In Chase Center",
"address":"4.5(21) · $$ · Coffee shop",
"thumbnail":"https://serpapi.com/searches/62bda4a2f0adfb0a9364cfb4/images/db978ef22623ebbaba5a259d7610a82f1e67d73393884509c7d902e95abeda94a9e2ac25fedcf5ad.jpeg"
}, ... other results
]
},
"organic_results":[
{
"position":1,
"title":"Coffee - Wikipedia",
"link":"https://en.wikipedia.org/wiki/Coffee",
"displayed_link":"https://en.wikipedia.org › wiki › Coffee",
"thumbnail":"https://serpapi.com/searches/62bda4a2f0adfb0a9364cfb4/images/0293bd7aa87e25161235b6fe723a6c2dae65c3f9f164074026df984d8f3e5820.jpeg",
"snippet":"Coffee is a brewed drink prepared from roasted coffee beans, the seeds of berries from certain flowering plants in the Coffea genus. From the coffee fruit, ...",
"snippet_highlighted_words":[
"Coffee",
"coffee",
"coffee"
],
"sitelinks":{
"inline":[
{
"title":"Coffee bean",
"link":"https://en.wikipedia.org/wiki/Coffee_bean"
},
{
"title":"History",
"link":"https://en.wikipedia.org/wiki/History_of_coffee"
},
{
"title":"Coffee preparation",
"link":"https://en.wikipedia.org/wiki/Coffee_preparation"
},
{
"title":"Coffee production",
"link":"https://en.wikipedia.org/wiki/Coffee_production"
}
]
},
"rich_snippet":{
"bottom":{
"extensions":[
"Region of origin: Horn of Africa and ‎South Ara...‎",
"Color: Black, dark brown, light brown, beige",
"Introduced: 15th century"
],
"detected_extensions":{
"introduced_th_century":15
}
}
},
"about_this_result":{
"source":{
"description":"Wikipedia is a multilingual free online encyclopedia written and maintained by a community of volunteers through open collaboration and a wiki-based editing system. Individual contributors, also called editors, are known as Wikipedians. Wikipedia is the largest and most-read reference work in history.",
"icon":"https://serpapi.com/searches/62bda4a2f0adfb0a9364cfb4/images/0293bd7aa87e25161235b6fe723a6c2d8021773b9a274d74b2c9d60da12b643d997acd2685004b3a989713a5bd03429c.png"
},
"keywords":[
"coffee"
],
"languages":[
"English"
],
"regions":[
"the United States"
]
},
"about_page_link":"https://www.google.com/search?q=About+https://en.wikipedia.org/wiki/Coffee&tbm=ilp&ilps=ADNMCi0tVhSB-fGHOJYgrIxB0xlXYrPGPA",
"cached_page_link":"https://webcache.googleusercontent.com/search?q=cache:U6oJMnF-eeUJ:https://en.wikipedia.org/wiki/Coffee+&cd=13&hl=en&ct=clnk&gl=us",
"related_pages_link":"https://www.google.com/search?gl=us&hl=en&q=related:https://en.wikipedia.org/wiki/Coffee+Coffee"
}, ... other results
],
... and other search results
}
Disclaimer, I work for SerpApi