Is there a generally available HTTPS POST smoke test? - language-agnostic

When debugging an HTTP client, one of your first tests is likely to be a Google search, which lets you see whether your client does non-SSL GETs properly. Everyone knows where it is, everyone can use it, and everyone can see whether it succeeded.
My client has a problem with HTTPS POST. I can reproduce it locally with my specially set-up HTTPS server, but I want others to be able to try it as well. Is there a public web page using HTTPS where sending a test POST is not a bad idea?
Edit: In the end, the problem turned out to be that my client would cache network output by the line when sending over TLS. Obviously, that causes problems for POST but not for GET...

I stumbled across this question while looking for the same thing, but I also found https://posttestserver.com/, which provides such a service for HTTP and HTTPS.

Google App Engine has supported HTTPS for a while now. That should give you a simple, easy way to put up test pages for anybody to see and serve them up over HTTP and HTTPS. Give us the link too, could be useful to use for our clients if the tests are generic enough.

The simplest public HTTPS post test I can think of would be webmail.
For example create a dummy Google account, then take the username and password of that dummy and see if the user can login using https://www.google.com/accounts/ManageAccount (a simple HTTPS post form).

Create a twitter account. Because of the json api, checking for a valid post to twitter is very simple. For the POST, you can look at the API docs for Status Updates. Once you've made a post, you can check the results at the User Timeline.
The API docs even have simple examples with curl to show you how easy it is. The POST:
curl -u user:password -d "status=testing my HTTP POST request" https://twitter.com/statuses/update.json
And getting the status to check it:
curl https://twitter.com/statuses/user_timeline.json?screen_name=user

Any login form should do.

In short, no. But without further info as to what specific bug you're experiencing, it's hard to search for something that already exists. My suggestion would be to find a free hosting service, and put the test page up there, along with a small google ad for some revenue.
Interesting concept, though, the publicly available test cases for standards. I like.

I'll bet that google search will accept a search paramter as POST if you sent it that way.

SSL adds a lot of complexity to the transaction, and you actually should break it up into two pieces.
You should do an GET w/ HTTPS first. When I was smoke testing networking for Netscape/AOL/Mozilla, I used http://www.verisign.com, because that was the home page for the main certificate vendor. I did not test the HTTP/SSL implementation itself, but we figured that while we are sitting around clicking on links in a build, we may as well do some SSL versions of the HTTP requests.
I cannot easily think of a good https: URL that uses POST, but I actually think it matters a lot less.
Once you know that SSL is working w/ HTTP at all, failures that are request-specific are going to be pretty limited, based on my recollection. Then again, this area was not assigned directly to me, so take that with a grain of salt.
My more recent thinking about testing is that test groups need to setup their systems, especially test servers. You would probably get better mileage by getting a good working set of instructions on how to configure HTTPS w/ a self signed certs, and then create your own internal POST test pages.

Related

Node.js security

I am building a basic app using node.js and mysql and just getting a hang of it, I would like to secure the api,like only allow certain people to access the data, maybe by passing a certain token each time a user requests for some information, I tried searching for certain tutorials which used node.js and mysql database and security, I am confused as to which security measure to use, I even read about Json Web Tokens but din't find a proper tutorial for that.Please point me in the right direction.
EDIT
What I meant to say was, only authenticated users are allowed to get access to data in the api, when a random visitor tries to access a URL he shouldnt be alowed to without proper authentication,what I am mostly looking for now is when a user is authenticated the user should be a sent a token of some sort so then gets access to private data,I don't exactly know how to go about this whole thing.Would be glad if you could clear it up for me.
I would look at implementing Oauth2 server in your app. I found this article useful:
http://blog.papersapp.com/oauth-server-in-node-js/
No sure what your exact question is about, but I think the below will help:
Node.js security tips: http://blog.risingstack.com/node-js-security-tips/
Secure Express apps with various HTTP headers: https://github.com/helmetjs/helmet
Go on an educational Web security adventure: https://github.com/toolness/security-adventure
Node.js Security presentation: http://www.slideshare.net/d0cent/nodejs-security?qid=c450507b-e491-4e9a-9b05-89d0c82ea10b&v=default&b=&from_search=6
Take a look at http://passportjs.org/ . Passport has support for alot of authentication methods, however, for your API, you will probably want to use OAuth (http://passportjs.org/docs/oauth2-api). OAuth is what most popular APIs use to authenticate consumers.
For simple projects, You can also use basic authentication, which is what you see when you see the browser prompt asking for username and password. This authentication information can be sent in the header when API consumers makes requests.

How can I retrieve HTTP responses from recorded log without receiving actual response from the server?

I think this is how should it work: I record the HTTP activity while browsing my website online. Then, When running my website's HTML offline, no network activity is made. And when a request is made, a response is retrieved from the recorded log.
I need to know how can I make this.
I think this tool is little near from what I want to do. But, unfortunately I don't know how to use it for my case.
The fact that you tagged this post with Fiddler suggests that you know that Fiddler can already do this using the AutoResponder tab. You may wish to update your question to explain why you don't want to do that.

Web crawler dealing with "Sign up or log in to read full content"

Given a page like this, I am trying to extract all the answer text with a ruby web crawler.
I am using nokogiri and search('div[#class="answer_content"]').inner_text to access the answers, but I can't seem to access all the text, even when in fact I am logged in. About 200 words down, I'll get the message "sign up or log in to read full content."
Also, is this div class the correct one to use?
It seems to me that you need to authenticate yourself from the crawler. I've done it a few weeks ago. I used a firefox extension called Tamper Data which allowed me to see the requests made between the browser and the server. In my case, the authentication was handled by a session id; I just had to get it back and pass it to each request I made to the server.
But in your case, the authentication might be done by a different way, you'll have to see for yourself. Anyway, I can detail if it's not clear enough.

Oauth for open source code

I am looking at creating a good google chrome extension for tumblr.
Now tumblr uses Oauth as it's authentication method.
So my question becomes.
Do I:
Distribute a single key with the application and hope people are nice enough to not use it for illegitimate purposes.
Put in the options the ability for a user to enter their own key, hence asking each user to register an application.
Create an intermediary on my server that makes the actual request.
Something else?
I also had to consider this for my Template extension when using OAuth to communicate with the goo.gl URL shortener service on the user's behalf.
After reading Chrome's OAuth tutorial I managed to implement this simply using the following;
ChromeExOAuth.initBackgroundPage({
access_url: 'https://www.google.com/accounts/OAuthGetAccessToken',
app_name: chrome.i18n.getMessage('app_name'),
authorize_url: 'https://www.google.com/accounts/OAuthAuthorizeToken',
consumer_key: 'anonymous',
consumer_secret: 'anonymous',
request_url: 'https://www.google.com/accounts/OAuthGetRequestToken',
scope: 'https://www.googleapis.com/auth/urlshortener'
});
This is surrounded by quite a bit of logic due to the complexity of the extension but this results in the user simply being shown a page (on a new tab) the first time they attempt to shorten a URL using my goo.gl implementation. Here the user can decide to allow/deny the authentication. Since this is only done once (unless the user later removes it from their list of authorized applications) I thought this was fine and my users seem to agree.
I did also provide an option to toggle OAuth on/off but I'm guessing this will always be required for the tumblr API. On that note I must admit I'm not familiar with that particular API so I'm afraid I can't help you further.

How to learn about web communcation standards (xauth rest soap oauth json)?

I am an android application developer (trying to be one) and I want to learn how to communicate with web sites from my applications. However, I don't know where to start. Can you point me in the right direction?
PS : I believe "web communcation standarts" is wrong term but i don't know how to call them.
I'd suggest that you learn by doing. Pick an idea - let's say you want to upload a picture from your Android phone to facebook.
Now go and research how to do that. Facebook exposes what they call the "Graph API" for all fb resources, including users, statuses, and photos, among other things. The Graph API is a REST API - in other words each item on the backend is modelled as a "resource" which is accessible from a URL. (Learn more about REST on wikipedia). Send a GET HTTP message to the given URL and you can get information on the resource. Send a POST to the container of that resource, and you can add another of whatever it is. In this model, to add a photo, you POST to the album URL. Read on FB for how to do that. They call it "publishing".
But reading that doc page you learn that in order to POST you need an authorization token.
Your app needs publish_stream permissions. How do you get that? Keep reading in the FB doc and you'll see a link to the Authentication and authorization guide. That tells you about how to use OAuth 2.0 in Facebook apps.
Your app will send its first feeble message out, and get an error response. Why? You'll want to examine the message. so you google and figure out that you can use an http debugging proxy to help you, something like Fiddler2 for Windows, or Charles for other systems. So you download that, set the proxy on your Android device, and now you can view outgoing HTTP messages and their responses on your PC.
Keep plugging along, learning what you need to accomplish the task in front of you: in this example, posting a photo from an android device to Facebook.
After you reach that goal, you will have learned quite a bit of practical knowledge for this domain of problem. Then you can try a new goal, and work on that.
After a few iterations of this, you will have learned how to learn, which is even better than learning REST or OAuth. If you know how to learn, the sky's the limnit.
Learn about HTTP first. There is far more to HTTP than there first appears. It is a building block for everything on the web.
You will not go wrong by getting very familiar with this http://www.w3.org/Protocols/rfc2616/rfc2616.html