Google, Twitter, Facebook all seem to have their own styles, colors and worst - sizes. Is there any way to use custom images WHILE also retaining the share 'counts'? A simple API or workaround for all 3 services?
These API's usually render markup to your page which can be targeted & styled through CSS. So, implementing your own CSS that targets the rendered markup may be the simplest and cleanest approach.
Another option is to create your own HTML, CSS and JavaScript that acts as a fascade between the underlying API (Facebook, G+, Twitter) and the client.
These will return simple JSON responses with counts for property of re-Tweets & Likes for a specific URL:
http://urls.api.twitter.com/1/urls/count.json?url=SOME_URL_HERE
http://graph.facebook.com/SOME_URL_HERE
Examples:
TWITTER API CALL:
http://urls.api.twitter.com/1/urls/count.json?url=http://stackoverflow.com
:: JSON RESPONSE:
{"count":4504,"url":"http://stackoverflow.com/"}
FACEBOOK GRAPH CALL:
http://graph.facebook.com/http://stackoverflow.com
:: JSON RESPONSE:
{
"id": "http://stackoverflow.com",
"shares": 7004,
"comments": 3
}
Currently there is no public API method for Google...
Related
I'm trying to write a spider that will automatically log in to this website. However, when I try using scrapy.FormRequest.from_response in the shell I get the error:
No <form> element found in <200 https://www.athletic.net/account/login/?ReturnUrl=%2Fdefault.aspx>
I can definitely see the form when I inspect element on the site, but it just did not show up in Scrapy when I tried finding it using response.xpath() either. Is it possible for the form content to be hidden from my spider somehow? If so, how do I fix it?
The form is created using Javascript, it's not part of the static HTML source code. Scrapy does not parse Javascript, thus it cannot be found.
The relevant part of the static HTML (where they inject the form using Javascript) is:
<div ng-controller="AppCtrl as appC" class="m-auto pt-3 pb-5 container" style="max-width: 425px;">
<section ui-view></section>
</div>
To find issues like this, I would either:
compare the source code from "View Source Code" and "Inspect" to each other
browse the web page with a browser without Javascript (when I develop scrapers I usually have one browser with Javascript for research and documentations and another one for checking web pages without Javascript)
In this case, you have to manually create your FormRequest for this web page. I was not able to spot any form of CSRF protection on their form, so it might be as simple as:
FormRequest(url='https://www.athletic.net/account/auth.ashx',
formdata={"e": "foo#example.com", "pw": "secret"})
However, I think you cannot use formdata, but instead they expect you to send JSON. Not sure if FormRequest can handle this, I guess you just want to use a standard Request.
Since they heavily use Javascript on their front end, you cannot use the source code of the page to find these parameters either. Instead, I used the developer console of my browser and checked the request/response that happened when I tried to login with invalid credentials.
This gave me:
General:
Request URL: https://www.athletic.net/account/auth.ashx
[...]
Request Payload:
{e: "foo#example.com", pw: "secret"}
Scrapy has a JsonRequest class to help with posting JSON. See here [https://docs.scrapy.org/en/latest/topics/request-response.html]
So something like the below should work
data = {"password": "pword", "username": "user"}
# JSON POST to API login URL
return JsonRequest(
url=url,
callback=self.after_login,
data=data,
)
I have been trying to get the carousel list on Skype. I have used the JSON specified in
https://dialogflow.com/docs/reference/api-v2/rest/v2beta1/projects.agent.intents#CarouselSelect
but the Skype isn't rendering it or the facebook messenger. If I'm sending a list of cards using
https://dialogflow.com/docs/reference/api-v2/rest/v2beta1/projects.agent.intents#Card
then the Skype is rendering a vertical listview and Messenger is rendering a carousel list. And the carousel select is working fine in V1 of Dialogflow using the message object
https://dialogflow.com/docs/reference/agent/message-objects#custom_payload_message_object_2 using which I'm able to send a custom payload to get the individual platforms, and also sending it in a payload in V2 didn't help.
Is there's a way to achieve the carousel list in Skype using DialogFlow V2? If it's possible to make this happen using payload, please reply with the JSON.
Thanks in advance!
Finally, I cracked it!
The default list view in messenger in a carousel and the default list view of Skype is normal vertical list view. For most of the official documents that I have seen for Skype bot development, Skype encourages its developers to use defined libraries using which the JSON is created and sent to the bot. Messenger, on the other hand, has many resources on the JSON format.
in DialogFlow V1:
In the message object, we get to add the details like the type of payload and the platform we are expected to send it to. More can be read at
https://dialogflow.com/docs/reference/agent/message-objects
So, the carousel in Skype is achieved by adding the Skype bot's attribute attachmentType
This specifies if the item is of carousel else is of normal list view.
The JSON format is specified at
https://miningbusinessdata.com/dialogflow-api-ai-skype-integration/
This is how a custom payload data is sent to Skype bot. Here, we can send a list of items in attachments array with the type and Skype shall interpret it.
in DialogFlow V2-Beta:
There are 2 ways I have identified using which we can send data to Skype/Messenger using V2-Beta.
Using the predefined templates given by dialogflow
https://dialogflow.com/docs/reference/api-v2/rest/v2beta1/projects.agent.intents#Message
This provides the card, image, carousel views. When a list of card objects is sent to Skype, it displays it as a vertical list and when sent to Messenger, it displays it as Carousel list. But, the Skype and Messenger didn't seem to support the carousel template
So the other option we are left with is sending data in the custom payload.
For sending it in a custom payload, there are 2 payloads in V2.
Inside Message object of the webHookResponse
https://dialogflow.com/docs/reference/api-v2/rest/v2beta1/projects.agent.intents#Message
The other payload is in the webHookResponse we shall send.
https://dialogflow.com/docs/reference/api-v2/rest/v2beta1/WebhookResponse
Both of these are JSON objects and sending it in the Message object actually works and with a proper JSON format. The JSON format of the Skype when being sent as a custom payload has very fewer resources available.
"platform":"SKYPE",
"payload":
{
"skype":
{
"attachmentLayout":"carousel",
"attachments":[
{
"contentType":"application/vnd.microsoft.card.hero",
"content":{
"title":
"subtitle":
"images":[{"url":}],
"buttons":[{
"type":"postBack",
"title":
"value":
}]
}
]
}
}
Using this the list of cards are sent to Skype and the list is displayed as Carousel list. On later searching I found out
https://tsmatz.wordpress.com/2016/08/31/microsoft-bot-framework-messages-howto-image-html-card-button-etc/
It has the list of different JSONs that we can send to Skype
Happy Coding!
Is it possible to set the Rule Lines options, under the View tab using the OneNote api.
I've had a look at the page and content end points but can't see anything that suggests itself.
It is possible to achieve this when creating a new page via the api. If you have a template page that is ruled and use page: copyToSection the resultant page will also be ruled.
POST https://graph.microsoft.com/beta/me/notes/pages/<id>/copyToSection
Content-type: application/json
Content-length: 52
{
"id": "id-value",
"groupId": "groupId-value"
}
Posting Yanir's comment as an answer:
This option is currently not supported through the API. I would encourage you to submit your idea to: onenote.uservoice.com/forums/245490-onenote-developer-apis, so we can explore working on it.
I have a list of web domains and would like to check if they are built to be mobile-responsive. A fairly sure way to check this manually is to see if there are "#media" queries in the style.css.
I've used XPATH (IMPORTXML) previously to bulk-check for strings on webpages, but I don't see an obvious way of importing the css files in bulk and search for a string within them. Is there a way to do this? Ideally, I'd like to accomplish it in Google Sheets or with Google Apps Script.
Thank you!
You can use Google's Mobile-Friendly Test if you want to use a GUI.
If you want to use a REST API, try this (replace url parameter for what you want to test):
https://www.googleapis.com/pagespeedonline/v3beta1/mobileReady?url=http://facebook.com
This will return a JSON object. It will return lots of useful info, but if you are just looking for mobile friendliness, look for the true or false result here:
"ruleGroups": {
"USABILITY": {
"pass": true
}
Hope that helps!
I have a form on my website which forwards the textbox input to www.google.com/search?q=.... Is there any possibility to get this autocomplete / autofill features google provides when searching over google.com. Firefox uses google's autocomplete function in the search bar next to the url bar, but how can I get this kind of information for my web form?
There was the Google Suggest API that allowed to use autocomplete on custom keywords. But it seems google has removed it. But I found out that you can still access the API. There are URLs for JSON and XML.
JSON
http://clients1.google.com/complete/search?hl=en&q=<your query here>&client=hp
XML
http://clients1.google.com/complete/search?hl=en&q=<your query here>&output=toolbar
Without any warranty.
If you want to get autocomplete on a text box. You can combine that with jQuery UI Autocomplete
Here an complete example with google auto suggest
http://googlesuggest-jquery.googlecode.com/svn/trunk/
The new url is:
http://suggestqueries.google.com/complete/search?client=firefox&q=YOURQUERY
the client part is required; I did't test other clients.
In addition to RickyA´s answer you can filter the suggestion by media using bs parameter:
http://clients1.google.com/complete/search?hl=en&q=stackoverflow&output=toolbar&ds=
youtube: ds=yt
books: ds=bo
products: ds=sh
news: ds=n
images: ds=i
web: ds=
recipes: ds=r