I've been testing out my data on this page:
https://developers.google.com/webmaster-tools/search-console-api-original/v3/searchanalytics/query
The data I am receive seems to be off compared to the data I am showing in google search console. I am wondering if this may be the result of a sampling error or something incorrect with my query. Any input on what may cause the data to change from Google Search Console to the Query result?
{
"startDate": "2019-04-01",
"endDate": "2019-04-30",
"dimensions": [
"query"
],
"searchType": "web",
"dimensionFilterGroups": [
{
"groupType": "and",
"filters": [
{
"dimension": "page",
"operator": "contains",
"expression": "/about/who-we-are/community/"
}
]
}
]
}
Data from query does not match Google search Console.
Example:
Google Search Console
"community"
Clicks 8
Impressions 13
Query
"community"
],
"clicks": 17.0,
"impressions": 26.0,
There are indeed some discrepancies between search console UI and API. I have similar problems and i found these documentation from Google:
https://support.google.com/webmasters/answer/7042828
https://support.google.com/webmasters/answer/7576553?hl=en&visit_id=637099209145003362-895149723&rd=1
Google has a certain aggregation method in which if you query from API in combination with page grouping the result will be different.
From UI:
If one query returned your property at positions 2, 4, and 6, its position is counted as 2 (the topmost position).
If a second query returned your property at positions 3, 5, and 9, its position is counted as 3 (the topmost position).
The average position across these two queries is (2 + 3)/2 = 2.5.
Related
I just trained my LUIS application and published it to production. If I test it on an utterance, I can see how that result compares to the published version and look at the JSON result. The problem is I'm getting a completely different JSON result there than I get when I query the API via its URL. Here is the test result JSON:
{
"query": "please show me *johnson*",
"prediction": {
"normalizedQuery": "please show me *johnson*",
"topIntent": "Show",
"intents": {
"Show": {
"score": 0.985523641
}
},
"entities": {
"ShowObject": [
"*johnson*"
],
"$instance": {
"ShowObject": [
{
"type": "ShowObject",
"text": "*johnson*",
"startIndex": 15,
"length": 9,
"score": 0.8382344,
"modelTypeId": 1,
"modelType": "Entity Extractor",
"recognitionSources": [
"model"
]
}
]
}
}
}
}
and here is the API query result:
{
"query": "please show me *johnson*",
"topScoringIntent": {
"intent": "Show",
"score": 0.985523641
},
"intents": [
{
"intent": "Show",
"score": 0.985523641
}
],
"entities": [
{
"entity": "* johnson *",
"type": "ShowObject",
"startIndex": 15,
"endIndex": 23,
"score": 0.8382344
}
]
}
The problem with the API query result is that it doesn't return enough information about the entity, and it returns a different entity than the test result. Note above that the test result returns *johnson* with no spaces near the asterisks, which is how the original query is, but the API query result returns * johnson * with spaces near the asterisks. I don't want it to put the spaces in, so I prefer the test result over the API query result.
Why are they different, and how do I get the API query to return a result like the test, i.e. with no modification of the input string to add spaces near the asterisks.
Here is the API query URL including parameters:
https://westus.api.cognitive.microsoft.com/luis/v2.0/apps/[app ID removed]?q=please+show+me+*johnson*&timezoneOffset=0&verbose=true&spellCheck=false&staging=false
Oh, I see now - this is probably to represent a wildcard search, right? If so, I'm not personally aware of a way to strip this out of a LUIS response, and I've seen similar, I think, when there's an #mention in there as well. However, if this is to facilitate searches, such that you know there's a good chance to have a * before and/or after the "ShowObject" entity, then it should be easy enough to test for this and replace, either string or regex (replace the "star-plus-space" with a space, I mean - I realise you need the star itself). Basically, you'd replace "[start][space]" with "[start]", and same at the end. Not pretty, but workable and simple to implement...
Just out of interest, do you anticipate * in the middle of a string as well?
I have a question about Json extractor sampler in Jmeter.
I have a json, that include list of components, the problem is that the order of the list is changing, and if I send the request in the morning the order of the list is not as in the evening .
I want to extract the value "SalesPerson" with Id = 10606.
The problem is that if the Id of 10606 is the first in response it is OK, But if it is in different order it brings wrong value.
I need that in each time I send a request I can search Id 10606 and return the sales person (in the example "Bebeto").
Is it can be done? that it will extract exactly the specific value and not according it's place in response regards.
{
"Error": null,
"ErrorCode": 0,
"Data": [{
"Account": "Pro",
"SalesPerson": "Ronaldo",
"Id": 7722,
"Name": "Brazil"
}, {
"Account": "Basic",
"SalesPerson": "Bebeto",
"Id": 10606,
"Name": "USA"
}, {
"Account": "Basic",
"SalesPerson": "Rivaldo",
"Id": 13017,
"Name": "Greece"
}],
"Totals": 3
}
Provided the sampler, that worked OK just if 10606 is the first in the list. I want it to extract Salesperson with Id 10606 regardless its position in the response. I want to provide 10606 and get the salesperson.
In the provided scenario the results is wrong since its extract "Ronaldo"
What am I missing?
Go for Filter Operator, something like:
$..[?(#.Id == '10606')].SalesPerson
should do the trick for you
Demo:
More information and tricks: JMeter's JSON Path Extractor Plugin - Advanced Usage Scenarios
I tried this query to get the list of all dish instances
[{
"id": null,
"name": null,
"type": "/food/dish"
}]
But it only gives me first page:
http://www.freebase.com/query?autorun=1&q=%5B%7B%22id%22:null,%22name%22:null,%22type%22:%22/food/dish%22%7D%5D
Question 1: How to add paging to get all 2.5K or so dish instances? I tried to add "cursor: 2" and didn't work.
Let say I have a name "pizza", I tried this query to get detail of "pizza":
{
"*": null,
"name": "pizza",
"type": "/food/dish"
}
But that didn't give me description and images like this page http://www.freebase.com/m/0663v
Question 2: How to get all information, or at least description and image URLs like in the freebase page above?
Bonus: I tried to do everything via Freebase Node.js here https://github.com/spencermountain/Freebase.js
I suggest you separate this into 2 questions, so each has their own topic, and it is easier for future visitors to search.
That said:
Question 1
You can increase the number of results you get per page by adding limit: to your query. Regardless, you will have to use paging. To use paging, you need to add the cursor parameter to your mqlread HTTP request. Again: cursor is not part of the MQL query itself, but rather of the HTTP envelope that submits it.
For the first query, issue an empty cursor, and for subsequent queries use the cursor value returned to you by mqlread.
Note that all this will need to be done with the API, not with freebase directly, and as such the URL will need to be:
https://www.googleapis.com/freebase/v1/mqlread?cursor=&query=[{"id":null,"name":null,"type":"/food/dish","limit":5}]
Also note that if you plan on doing this for anything other than testing you will need to obtain a key from Google.
Finally, note that some strings in Freebase are "freebase-encoded", read up on that regarding how to decode them in the result.
Question 2
If you just want the ingredient names then simply add "/dining/cuisine/ingredients": [] to your query. Note that many dishes do not have ingredients, but Pizza does:
{
"id": "/m/0663v",
"name": null,
"type": "/food/dish",
"/dining/cuisine/ingredients": []
}
Getting the images means adding "/common/topic/image": [{}] to your query, and using the returned id for each image.
Getting an image URL from a given image id is done by prepending https://usercontent.googleapis.com/freebase/v1/image/ to the id.
Edit
Tom correctly noted that I forgot about image descriptions. The description for each image will be available under name: in the returned /common/topic/image array. For example, for the query
[{
"id": "/en/minestrone",
"/common/topic/image": [{
"id": null,
"name": null
}]
}]
you get the following images and their descriptions:
{
"result": [{
"id": "/en/minestrone",
"/common/topic/image": [
{
"id": "/wikipedia/images/commons_id/1492185",
"name": "MinestroneSoup"
},
{
"id": "/wikipedia/images/commons_id/12565901",
"name": "Homemade minestrone"
}
]
}]
}
Your final MQL, then, is:
[{
"id": null,
"name": null,
"type": "/food/dish",
"/common/topic/image": [{
"id": null,
"name": null
}],
"/dining/cuisine/ingredients": []
}]
... and the HTTP envelope will contain a key and a value for cursor.
See Nitzan's answer for the answer to question 1.
For your second question, the easiest way to get descriptions and images is by using the Topic API e.g. https://www.googleapis.com/freebase/v1/topic/m/0663v
I need to crawl a paginated REST API with offset and limit parameters using Talend.
The API gives me a list of the resources I am interested in.
For instance, the response to the initial request with offset=0 and limit=2 is:
{
"meta": {
"limit": 2,
"next": "/api/v1/request/?offset=2&limit=2",
"offset": 0,
"previous": null,
"total_count": 4300
},
"objects": [
{
"id": 1,
"name": "foo"
},
{
"id": 2,
"name": "bar"
}
]
}
As you can see, the response object contains an objects key, i.e. some of the desired resources and a meta key which indicates the next URL to query: next. So far I am able to perform the initial request with tRESTClient. However, I don't know how to proceed from here and request the remaining pages using the clue given by next.
How can I perform multiple requests to that API so that I iterate over the whole list until next equals null (=list is exhausted)?
I tried to figure out how tSetGlobalVar and tLoop could help me, but so far with no success. But then again, I am a Talend newbie.
Current job
This is what my job currently looks like:
I'm trying to figure out how to write a MQL query to get a list of all the types associated to a given word.
For example I tried:
{
"id":null,
"name":null,
"name~=": "SOME_WORD",
"type":"/type/type",
"domain": {
"id": null,
"/freebase/domain_profile/category": {
"id": "/category/commons"
}
}
}
I found this to list out all the Commons types or categories but haven't yet figured out how to narrow it down for a given input.
[{
"id": null,
"name": null,
"type": "/freebase/domain_profile",
"category": {
"id": "/category/commons"
}
}]
There are a couple of different ways to do this with different tradeoffs for each.
Use the Search API with a query like this
https://www.googleapis.com/freebase/v1/search?indent=true&filter=%28all%20name{full}:%22uss%20constitution%22%29
You'll get back JSON results which look like this:
{
"status": "200 OK",
"result": [
{
"mid": "/m/07y14",
"name": "USS Constitution",
"notable": {
"name": "Ship",
"id": "/boats/ship"
},
"lang": "en",
"score": 1401.410400
},
...
You can make the matching more liberal by switching the "{full}" to "{phrase}" which will give you a substring match instead of an exact match.
Caveats:
- You'll only get a single "notable type" and it's fixed by Freebase's (unknown) algorithm
- I don't think there's a way to get both USS Constitution & U.S.S. Constitution results
- You can get a list of all types by adding &mql_output={"type":[]}, but then you lose the "notable" type. I don't think there's a way to get both in a single call.
Use MQL
This query returns the basic information that you want:
[{
"name~=":"uss constitution",
"type":[],
"/common/topic/notable_types" : []
}]
Caveats:
It won't find "uss constitution" which is an alias rather than the primary name (there's a recipe in the MQL cookbook for that though)
It won't find "u.s.s. constitution"
The "notable_types" API is an MQL extension and MQL extensions aren't supported in the new Freebase API, only the legacy deprecated API
You're tied to whatever (unknown) algorithm Freebase used to compute "notability"
Depending on what you are trying to accomplish, you might need something more sophisticated than this (as well as a deeper understanding of what's in Freebase), but this should get you going with the basics.
Did you try:
[{
"name": "David Bowie",
"type": []
}]