Twitter endpoint request gets HTML response in Node app deployed to Heroku - json

I made a small script querying a Twitter public endpoint to ask for a username's availability.
A cURL request to this endpoint throws a JSON response formatted like this:
$ curl 'https://twitter.com/users/username_available?username=jack'
{
"valid":false,
"reason":"",
"msg":"Nombre de usuario no disponible",
"desc":"Nombre de usuario no disponible"
}
Based on this, I wrote a small NodeJS app which makes this same request (using Request package).
Both CURL and my app work fine when run locally.
But in Heroku both CURL and my app get a full HTML response (with Twitter's mobile version) instead of JSON, so my app can't parse and evaluate.
Anyone had similar issues, and know how to get it working?
Thanks!
Edit (extra info):
Script is running in a web dyno in a free plan. Can't figure out how to switch to a worker dyno (if that's even still a thing)
Edit 2 (update): Now it's running on a worker Dyno, but the issue persists. Still getting HTML responses.

The endpoint you are calling is not a documented or supported Twitter API endpoint, so there should be no expectation of support, or guarantee that it will work. If you try to hit such an endpoint regularly from an unknown IP address then you should expect the IP to be blocked by Twitter's infrastructure as a protective measure.

Related

Getting "The server encountered an error processing the request. See server logs for more details." when sending a POST via the Chrome Postman plugin

I have been looking for a way to understand the root cause of the "The server encountered an error processing the request. See server logs for more details." problem in my particular case but none of the existing online solutions addresses the problem I am about to describe.
I need to send POST requests with JSON payload {"userName", "Adnan"} to an Azure-based web service which eventually retrieves some database GPS records in a HTTP response. My initial attempt to test this POST request worked fine via Fiddler (See image below) and Android OKHttp libraries.
However, when I tested the same request over my ultimate 3G platform, it gave the above-mentioned error. I managed to replicate the same error with the Chrome Postman plugin (See image below). It must be noted that even if I am sending the data as a JSON payload, the same error appears on the Postman plugin. Almost all the solutions to this problem online suggest on tweaking the webservice itself which should not be the case as it works fine with Fiddler and Android. So - my question is why the POST request shown above is not working with the Postman plugin. I believe if the problem is resolved on Postman, it would be resolved on my 3G embedded board as well. Thank you.

How does restangular talk to MySQL database

I am total JS newbie working on a project build in Grail 2.4.4, a web-app. It's a working app, build by a developer whom is not available anymore.
To get it to work locally I had to upgrade it to Grails 3.2.0. I got it almost working in Netbeans. But I got stuck at getting the data from the MySQL database.
The Chrome inspector says:
angular.min.js GET http://localhost:8080/<app>/currency/allCurrencies 404 ()
The controllers are written in Restangular which call the above URL.
What am I missing?
Firstly, Restangular is an Angular library which simplifies and standardizes making calls to a REST backend (which in your case is a Grails app). So, Restangular does not directly retrieve data from your a database, it invokes a web service which (in some cases) may retrieve data from a database.
In your case, Restangular is attempting to retrieve data from the endpoint http://localhost:8080/<app>/currency/allCurrencies but you are getting a 404 response, indicating that there is no endpoint mapped to this URL.
HTTP REST helps you connect to the API easily. Restangular can handle that by sending standard methods [Get, Post, Delete, Put] to the api like what you see.
This mean StudentController > Get()
localhost:2045/api/student
This mean StudentController > Get(Guid id)
localhost:2045/api/student/8ae37cfa-905b-4c71-ad03-bf416d93bdf8
This mean StudentController > POST(Guid id) ... if you send Post method to the API, it will detect it, this work also on put method
localhost:2045/api/student
use this module to get easily rest api.
Http-Rest-Service

implementing REST sdk for custom platform

I am implementing paypal REST integration for a new platform
I got stuck and would apprecıate any help
The interesting problem I get is the REST request I copied from the documentation page gives error. https://developer.paypal.com/docs/rest/api/payments/
I am not using command line but a little windows app to send web requests.
I just used exact same payload and got this error
The remote server returned an error: (503) Server Unavailable.
{"name":"INTERNAL_SERVICE_ERROR","information_link":"https://api.sandbox.paypal.com/docs/api/#INTERNAL_SERVICE_ERROR","debug_id":"c4a8716d8a791"}
The C# sample from the github works and when I use th json it generated everything works ok.
I found out that if I change the cardnumber the sample works. Will let paypal team now.

What's wrong with this authorization exchange?

I've set up a MediaWiki server on an Azure website with the PluggableAuth and OpenID Connect extensions. The latter uses the PHP OpenID Connect Basic Client library. I am an administrator in the Azure AD domain example.com, wherein I've created an application with App ID URI, sign-on URL and reply URL all set to https://wiki.azurewebsites.net/. When I navigate to the wiki, I observe the following behavior (cookie values omitted for now):
Client Request
GET https://wiki.azurewebsites.net/ HTTP/1.1
RP Request
GET https://login.windows.net/example.com/.well-known/openid-configuration
IP Response
(some response)
RP Response
HTTP/1.1 302 Moved Temporarily
Location: https://login.windows.net/{tenant_id}/oauth2/authorize?response_type=code&redirect_uri=https%3A%2F%2Fwiki.azurewebsites.net%2F&client_id={client_id}&nonce={nonce}&state={state}
Client Request
(follows redirect)
IP Response
HTTP/1.1 302 Found
Location: https://wiki.azurewebsites.net/?code={code}&state={state}&session_state={session_state}
Client Request
(follows redirect)
RP Request (also repeats #2 & #3)
POST https://login.windows.net/{tenant_id}/oauth2/token
grant_type=authorization_code&code={code}&redirect_uri=https%3A%2F%2Fwiki.azurewebsites.net%2F&client_id={client_id}&client_secret={client_secret}
IP Response
(As interpreted by MediaWiki; I don't have the full response logged at this time)
AADSTS50001: Resource identifier is not provided.
Note that if I change the OpenID PHP client to provide the 'resource' parameter in step 8, I get the following error response from AAD instead:
RP Request
POST https://login.windows.net/{tenant_id}/oauth2/token
grant_type=authorization_code&code={code}&redirect_uri=https%3A%2F%2Fwiki.azurewebsites.net%2F&resource=https%3A%2F%2Fwiki.azurewebsites.net%2F&client_id={client_id}&client_secret={client_secret}
IP Response
AADSTS90027: The client '{client_id}' and resource 'https://wiki.azurewebsites.net/' identify the same application.
(This has come up before.)
Update
I've made some progress based on #jricher's suggestions, but after working through several more errors I've hit one that I can't figure out. Once this is all done I'll submit pull requests to the affected libraries.
Here's what I've done:
I've added a second application to the example.com Azure AD domain, with the App ID URI set to mediawiki://wiki.azurewebsites.net/, as a dummy "resource". I also granted the https://wiki.azurewebsites.net/ application delegated access to this new application.
Passing in the dummy application's URI as the resource parameter in step #8, I'm now getting back the access, refresh, and ID tokens in #9!
The OpenID Connect library requires that the ID token be signed, but while Azure AD signs the access token it doesn't sign the ID token. It comes with the following properties: {"typ":"JWT","alg":"none"}. So I had to modify the library to allow the caller to specify that unsigned ID tokens are considered "verified". Grrr.
Okay, next it turns out that the claims can't be verified because the OpenID Provider URL I specified and the issuer URL returned in the token are different. (Seriously?!) So, the provider has to be specified as https://sts.windows.net/{tenant_id}/, and then that works.
Next, I found that I hadn't run the MediaWiki DB upgrade script for the OpenID Connect extension yet. Thankfully that was a quick fix.
After that, I am now left with (what I hope is) the final problem of trying to get the user info from AAD's OpenID Connect UserInfo endpoint. I'll give that its own section.
Can't get the user info [Updated]
This is where I am stuck now. After step #9, following one or two intermediate requests to get metadata and keys for verifying the token, the following occurs:
RP Request:
(Updated to use GET with Authorization: Bearer header, per MSDN and the spec.)
GET https://login.windows.net/{tenant_id}/openid/userinfo
Authorization: Bearer {access_token}
IP Response:
400 Bad Request
AADSTS50063: Credential parsing failed. AADSTS90010: JWT tokens cannot be used with the UserInfo endpoint.
(If I change #10 to be either a POST request, with access_token in the body, or a GET request with access_token in the query string, AAD returns the error: AADSTS70000: Authentication failed. UserInfo token is not valid. The same occurs if I use the value of the id_token in place of the access_token value that I received.)
Help?
Update
I'm still hoping someone can shed light on the final issue (the UserInfo endpoint not accepting the bearer token), but I may split that out into a separate question. In the meantime, I'm adding some workarounds to the libraries (PRs coming soon) so that the claims which are already being returned in the bearer token can be used instead of making the call to the UserInfo endpoint. Many thanks to everyone who's helped out with this.
There's also a nagging part of me that wonders if the whole thing would not have been simpler with the OpenID Connect Basic Profile. I assume there's a reason why that was not implemented by the MediaWiki extension.
Update 2
I just came across a new post from Vittorio Bertocci that includes this helpful hint:
...in this request the application is asking for a token for itself! In Azure AD this is possible only if the requested token is an id_token...
This suggests that just changing the token request type in step 8 from authorization_code to id_token could remove the need for the non-standard resource parameter and also make the ugly second AAD application unnecessary. Still a hack, but it feels like much less of one.
Justin is right. For authorization code grant flow, your must specify the resource parameter in either the authorization request or the token request.
Use &resource=https%3A%2F%2Fgraph.windows.net%2F to get an access token for the Azure AD Graph API.
Use &resource=https%3A%2F%2Fmanagement.core.windows.net%2F to get a token for the Azure Service Management APIs.
...
Hope this helps
Microsoft's implementation of OpenID Connect (and OAuth2) has a known bug where it requires the resource parameter to be sent by the client. This is an MS-specific parameter and requiring it unfortunately breaks compatibility with pretty much every major OAuth2 and OpenID Connect library out there. I know that MS is aware of the issue (I've been attempting to do interoperability testing with their team for quite a while now), but I don't know of any plans to fix the problem.
So in the mean time, your only real path is to hack your client software so that it sends a resource parameter that the AS will accept. It looks like you managed to make it send the parameter, but didn't send a value that it liked.
I had issues getting this running on Azure, even though I got something working locally. Since I was trying to setup a private wiki anyway, I ended up enabling Azure AD protection for the whole site by turning on:
All Settings -> Features -> Authentication / Authorization
From within the website in https://portal.azure.com
This made it so you had to authenticate to Azure-AD before you saw any page of the site. Once you were authenticated a bunch of HTTP Headers are set for the application with your username, including REMOTE_USER. As a result I used the following plugin to automatically log the already authenticated user into Azure:
https://www.mediawiki.org/wiki/Extension:Auth_remoteuser

Assistance required to connect to Foursquare via TIBCO BW using REST / JSON API - 1.1

We have an assignment to check the compatibility of Quick connect project for Foursquare against the REST/JSON 1.1.
When I am testing the process, while invoking Rest API for query Check-in, the process is throwing an error – “OAuth token invalid or revoked”. Can you kindly help to fix this issue.
Access_token for this was generated by registering a sample app with Foursuare.com, having www.google.com as welcome page and redirect URL. Please let me know if this is causing the issue.
Also, let me know is there is any standard method to generate the access_token for this.
Thanks in advance.
Regards,
Shree.
It sounds like you're not following Foursquare's instructions on connecting properly, or need to reconnect to get a new access_token. Keep in mind that the access_token isn't the code that comes back in the redirect, the code needs to be used to exchange for the access_token.