I have a Rails 5.0.0.1 API application that needs to return a simple PORO as json. I am using gem 'responders', '~> 2.3.0'
The object constructs correctly and works properly in the Rails server. It has only 4 attributes that are needed by the front-end.
The serializer is:
class WebsiteConfigSerializer < ActiveModel::Serializer
attributes :force_first_user_to_be_admin, :allows_delete_of_last_admin,
:is_private_website, :invite_only
end
The controller show method (the only method in the controller) is:
def show
config = WebsiteConfig.new
puts "Config: #{config.to_json}"
respond_to do |format|
format.json {render( json: config, status: 200 ) }
end
end
The log shows the request output, including the debugging puts statement:
Started GET "/config.json" for 127.0.0.1 at 2016-09-03 18:23:32 -0400
Processing by WebsiteConfigController#show as JSON
Config: {"force_first_user_to_be_admin":true, "allows_delete_of_last_admin":false, "is_private_website":true, "invite_only":true}
[active_model_serializers] Rendered WebsiteConfigSerializer with ActiveModelSerializers::Adapter::Attributes (0.13ms)
Completed 200 OK in 2ms (Views: 1.2ms | ActiveRecord: 0.0ms)
as expected.
The output of cURL is:
curl -sb -H "Accept:application/json" -H "Content-Type:application/json" http://localhost:3000/config
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx</center>
</body>
</html>
This is NOT what I expected.
wget output is EMPTY (0 bytes). Also not what I expected.
Obviously I am missing something. Any help will be appreciated.
Your request has been redirected, check if you have some authentication code like requiring login or something else, then added them to the curl command
Related
I'm trying to get some JSON data from this API - https://ped.uspto.gov/api/queries
This cURL request works fine and returns what is expected:
curl -X POST "https://ped.uspto.gov/api/queries" -H "accept: application/json" -H "Content-Type: application/json" -d "{ \"searchText\":\"*:*\", \"fq\":[ \"totalPtoDays:[1 TO 99999]\", \"appFilingDate:[2005-01-01T00:00:00Z TO 2005-12-31T23:59:59Z]\" ], \"fl\":\"*\", \"mm\":\"100%\", \"df\":\"patentTitle\", \"facet\":\"true\", \"sort\":\"applId asc\", \"start\":\"0\"}"
I have this python script to do the same thing:
from requests.structures import CaseInsensitiveDict
import json
url = "https://ped.uspto.gov/api/queries"
headers = CaseInsensitiveDict()
headers["accept"] = "application/json"
headers["Content-Type"] = "application/json"
data = json.dumps({
"searchText":"*:*",
"fq":[
"totalPtoDays:[1 TO 99999]",
"appFilingDate:[2005-01-01T00:00:00Z TO 2005-12-31T23:59:59Z]"
],
"fl":"*",
"mm":"100%",
"df":"patentTitle",
"facet":"true",
"sort":"applId asc",
"start":"0"
})
resp = requests.post(url, headers=headers, data=data)
print(resp.status_code)
but it returns a 403 error code and the following response header:
"Date":"Mon, 24 Oct 2022 16:13:58 GMT",
"Content-Type":"text/html",
"Content-Length":"919",
"Connection":"keep-alive",
"X-Cache":"Error from cloudfront",
"Via":"1.1 d387fec28536c5aa92926c56363afe9a.cloudfront.net (CloudFront)",
"X-Amz-Cf-Pop":"LHR50-P8",
"X-Amz-Cf-Id":"RMd69prehvXNAl97mo0qyFtuBIiY8r9liIxcQEmbdoBV1zwXLhirXA=="
I'm at quite a loss at what to do, because I really don't understand what my Python is missing to replicate the cURL request.
Thanks very much.
I was interested in this. I got an account with uspto.gov and acquired an access key. Their other API's work well. But the PEDS API? I kept getting the Cloudflare Gateway Timeout 503 error. While I was on their website, I looked into the PEDS API, I could not load any link to a https://ped.uspto.gov page.
I called them and they gave me an email address. I got this reply:
The PEDS API was taken down, because repeated data mining was bringing the entire PEDS System down.
The PEDS Team is working on a solution to fix the PEDS API, so that it can be re-enabled.
I tried it using PHP.
Cloudflare has been causing a lot of problems for curl.
I got a timeout.
I may have gotten past the 403 Forbidden, but did not have credentials and so the server dropped the connection.
An HTTP 504 status code (Gateway Timeout) indicates that when
CloudFront forwarded a request to the origin (because the requested
object wasn't in the edge cache), one of the following happened: The
origin returned an HTTP 504 status code to CloudFront. The origin
didn't respond before the request expired.
AWS Cloudflare Curl Issues
bypassing CloudFlare 403
How to Fix Error 403 Forbidden on Cloudflare
403 Forbidden cloudflare
██████████████████████████████████████████████████████████████
This is a conversion from you curl.
The Content-Type:application/data is added by default when you send JSON data.
I do not know about your json_data.dump or you putting the JSON in parentheses.
import requests
headers = {
'accept': 'application/json',
}
json_data = {
'searchText': '*:*',
'fq': [
'totalPtoDays:[1 TO 99999]',
'appFilingDate:[2005-01-01T00:00:00Z TO 2005-12-31T23:59:59Z]',
],
'fl': '*',
'mm': '100%',
'df': 'patentTitle',
'facet': 'true',
'sort': 'applId asc',
'start': '0',
}
response = requests.post('https://ped.uspto.gov/api/queries', headers=headers, json=json_data)
So I have a website that's fully working, with some URI encoded in the URL.
however, when I try to pass the URL to my chrome browser:
http://somewhere:3000/find/someOne?utf8=%E2%9C%93&search=someThing&choicen=no&querys={%22peopleName%22%3A%22%22%2C%22peopleGroup%22%3A%22%22%2C%22place%22%3A%22%22%2C%22pip%22%3A%22%22%2C%22hw%22%3A%22%22%2C%22somerock%22%3A%22%22%2C%22rocksomerock%22%3A%22%22%2C%22diedAt%22%3A%222016-01-01%20-%202016-12-31%22%2C%22borndAt%22%3A%22%22%2C%22taxRate%22%3A%22%22}
-- it throws me an error in the browser:
Bad Request
bad URI `/find/someOne?utf8=%E2%9C%93&search=someThing&choicen=no&querys={%22peopleName%22%3A%22%22%2C%22peopleGroup%22%3A%22%22%2C%22place%22%3A%22%22%2C%22pip%22%3A%22%22%2C%22hw%22%3A%22%22%2C%22somerock%22%3A%22%22%2C%22rocksomerock%22%3A%22%22%2C%22diedAt%22%3A%222016-01-01%20-%202016-12-31%22%2C%22borndAt%22%3A%22%22%2C%22taxRate%22%3A%22%22}'.
WEBrick/1.3.1 (Ruby/1.9.3/2014-11-13) at somewhere.com:3000
Also shows [2016-07-04 18:11:31] ERROR bad URI in the rails console
Versions:
rails3
Ruby 1.9.3
Any idea how to get it working? Is it because the { and } in the URI or because it is too long?
Parse the path in the controller upon incoming request, using Rack::Utils#parse_nested_query, see: http://www.rubydoc.info/github/rack/rack/master/Rack/Utils.parse_nested_query
# config/routes.rb
get '/find/someOne/*str' => 'find#someOne'
# app/controllers/find_controller.rb
class FindController < ApplicationController
def someOne
custom_params = Rack::Utils.parse_nested_query(request.env['ORIGINAL_FULLPATH'])
querys_hash = JSON.parse(custom_params["querys"])
end
end
Example via console:
$ bundle exec rails c
Running via Spring preloader in process 31944
Loading development environment (Rails 5.0.0)
irb(main):001:0> custom_params = Rack::Utils.parse_nested_query "utf8=%E2%9C%93&search=someThing&choicen=no&querys={%22peopleName%22%3A%22%22%2C%22peopleGroup%22%3A%22%22%2C%22place%22%3A%22%22%2C%22pip%22%3A%22%22%2C%22hw%22%3A%22%22%2C%22somerock%22%3A%22%22%2C%22rocksomerock%22%3A%22%22%2C%22diedAt%22%3A%222016-01-01%20-%202016-12-31%22%2C%22borndAt%22%3A%22%22%2C%22taxRate%22%3A%22%22}"
=> {"utf8"=>"✓", "search"=>"someThing", "choicen"=>"no", "querys"=>"{\"peopleName\":\"\",\"peopleGroup\":\"\",\"place\":\"\",\"pip\":\"\",\"hw\":\"\",\"somerock\":\"\",\"rocksomerock\":\"\",\"diedAt\":\"2016-01-01 - 2016-12-31\",\"borndAt\":\"\",\"taxRate\":\"\"}"}
irb(main):002:0> querys_hash = JSON.parse custom_params["querys"]
=> {"peopleName"=>"", "peopleGroup"=>"", "place"=>"", "pip"=>"", "hw"=>"", "somerock"=>"", "rocksomerock"=>"", "diedAt"=>"2016-01-01 - 2016-12-31", "borndAt"=>"", "taxRate"=>""}
I am using Frisby.js to submit a POST request with JSON data and test the JSON response. However, my query returns HTTP Status 500. The same POST request works fine in CURL in my command line, as well as Postman. What could be the problem?
CURL request: curl -H "Content-Type: application/json; charset=UTF-8" -X POST -d "json_data" url
FRISBY test:
frisby
.create("Submit request")
.post(url, json_data, {"json": true, "content-type": "application/json;charset=UTF-8"})
.expectStatus(200)
.inspectBody()
.toss();
OUTPUT with Frisby:
Destination URL may be down or URL is invalid, Error: ETIMEDOUT
F
Failures:
1) Frisby Test: Submit request
[ POST url]
Message:
Expected 500 to equal 200.
Stacktrace:
Error: Expected 500 to equal 200.
at null.<anonymous>
at null.<anonymous>
at Timer.listOnTimeout (timers.js:92:15)
Finished in 5.032 seconds
1 test, 1 assertion, 1 failure, 0 skipped
The 500 error and accompanying message that results "Destination URL..." is from Frisby itself, not your server. Frisby adds the error if the connection times out. Try increasing it to 10 seconds.
...
.get(url)
.timeout(10000)
...
I hope someone can share their knowledge with me. I have a small Sinatra app that I want to POST json data to it. The record is being created, thankfully, but none of the attributes make it. I've read several JSON examples using curl but they all yield the same result. So it makes me think it's my model. I can create records using tux fine so perhaps it's not my model.
curl command:
curl -XPOST -H "Content-Type: application/json"
"localhost:4567/date" -d
'{ "post": { "title": "different tile here"}}'
curl output:
=> {"id":13,"title":null,"body":null,"created_at":"2014-04-21T18:13:53Z",
"updated_at":"2014-04-21T18:13:53Z"}
sinatra output:
[2014-04-27T20:03:48.035710 #45360] DEBUG -- : (0.1ms) commit transaction
127.0.0.1 - - [27/Apr/2014 20:03:48] "POST /date HTTP/1.1" 200 - 0.0181
{"post":{"title":"different tile here"}}
D, [2014-04-27T20:09:32.614274 #45360] DEBUG -- : (0.1ms) begin transaction
D, [2014-04-27T20:09:32.615917 #45360] DEBUG -- : SQL (0.4ms) INSERT INTO "posts" ("created_at", "updated_at") VALUES (?, ?) [["created_at", 2014-04-28 01:09:32 UTC], ["updated_at", 2014-04-28 01:09:32 UTC]]
D, [2014-04-27T20:09:32.617656 #45360] DEBUG -- : (1.5ms) commit transaction
D, [2014-04-27T20:09:32.617852 #45360] DEBUG -- : (0.1ms) begin transaction
D, [2014-04-27T20:09:32.618132 #45360] DEBUG -- : (0.0ms) commit transaction
127.0.0.1 - - [27/Apr/2014 20:09:32] "POST /date HTTP/1.1" 200 - 0.0070
D, [2014-04-27T20:09:57.796909 #45360] DEBUG -- : Post Load (0.3ms) SELECT "posts".* FROM "posts" ORDER BY created_at DESC
127.0.0.1 - - [27/Apr/2014 20:09:57] "GET / HTTP/1.1" 200 2149 0.0128
app.rb
require 'sinatra'
require 'sinatra/activerecord'
require './environments'
require 'sinatra/flash'
require 'sinatra/redirect_with_flash'
require 'json'
class Post < ActiveRecord::Base
end
post "/posts" do
#post = Post.new(params[:post])
if #post.save
redirect "posts/#{#post.id}"
else
erb :"posts/create"
end
end
post '/date', :provides => 'json' do
data = JSON.parse(request.body.read)
json_data = data.to_json
content_type :json
##post = Post.new(params)
#post = Post.create(
title: data[:title]
body: data[:body]
)
if #post.save
#post.to_json
else
halt 500
end
end
get "/posts/create" do
#title = "Create post"
#post = Post.new
erb :"posts/create"
end
get "/posts/:id" do
#post = Post.find(params[:id])
#title = #post.title
erb :"posts/view"
end
get "/" do
#posts = Post.order("created_at DESC")
#title = "Welcome."
erb :"posts/index"
end
Gemfile
source 'https://rubygems.org'
ruby "2.0.0"
gem "sinatra"
gem 'activerecord', '4.0.4'
gem "sinatra-activerecord"
gem 'sinatra-flash'
gem 'sinatra-redirect-with-flash'
gem 'json'
group :development do
gem 'sqlite3'
gem "tux"
end
group :production do
gem 'pg'
end
environments.rb
configure :development do
set :database, 'sqlite3:///dev.db'
set :show_exceptions, true
end
configure :production do
db = URI.parse(ENV['DATABASE_URL'] || 'postgres:///localhost/mydb')
ActiveRecord::Base.establish_connection(
adapter: db.scheme == 'postgres' ? 'postgresql' : db.scheme,
host: db.host,
username: db.user,
password: db.password,
database: db.path[1..-1],
encoding: 'utf9'
)
end
I've left off my layout files since I'm testing this POST via this rudimentary API. I've bounced around the few examples of Sinatra API's on the web and don't know what I'm not seeing. What am I overlooking here? It's almost like Strong Parameters in Rails 4, but if so why does tux allow me to create records? thanx, sam
I'm using tastypie with django. I have one line of code:
data = self.deserialize(request, request.body, format=request.META.get('CONTENT_TYPE', 'application/json'))
I use this code from the command line to send a post request to my webserver:
curl -X post -d "{ 'username' : 'user', 'password' : 'password' }" http://127.0.0.1:8000/api/employee/login/ --header "Content-Type:application/json"
When I run this, it results in a json response of
{"error": ""}
Looking at my server logs I see:
[15/Feb/2014 20:39:49] "post /api/user/login/ HTTP/1.1" 400 13
A log message logged immediately before the deserialize line will be logged successfully, but a log message logged immediately after the deserialize line will not be logged, so I am pretty sure the deserialize is wrong. Does anyone know what could be wrong or if I should consider something else as the problem?
Your JSON is not valid. Please check it here. The 400 (bad request) status should give you clue about that. It should be: {"username": "user", "password": "password"}. Here you have some solutions how to escape " char in CURL command. Tastypie unfortunately raises exception without message here but we can easily fix that for future to save time for other people which will use your API.
from tastypie.exceptions import BadRequest
from tastypie.serializers import Serializer
class VerboseSerializer(Serializer):
"""
Gives message when loading JSON fails.
"""
# Tastypie>=0.9.6,<=0.11.0
def from_json(self, content):
"""
Override method of `Serializer.from_json`. Adds exception message when loading JSON fails.
"""
try:
return json.loads(content)
except ValueError as e:
raise BadRequest(u"Incorrect JSON format: Reason: \"{}\" (See www.json.org for more info.)".format(e.message))
class MyResource(BaseModelResource):
class Meta:
serializer = VerboseSerializer(formats=['json'])