Multipart upload to Google Bigquery - json

I'm trying to formulate a body for my load job (multipart) into Bigquery. Following the example on https://cloud.google.com/bigquery/loading-data-post-request#multipart:
-foo_bar_baz
Content-Type: application/json; charset=UTF-8
{
"configuration": {
"load": {
"sourceFormat": "NEWLINE_DELIMITED_JSON",
"schema": {
"fields": [
{"name":"word","type":"STRING"},
{"name":"word_count","type":"INTEGER"},
{"name":"corpus","type":"STRING"},
{"name":"corpus_date","type":"INTEGER"}
]
},
"destinationTable": {
"projectId":"myproject",
"datasetId":"mydataset",
"tableId":"mytable"
}
}
}
}
--foo_bar_baz
Content-Type: */*
[MY-JSON-DATA]
--foo_bar_baz--
But I get the message from google saying:
"message": "Job configuration must contain exactly one job-specific configuration object (e.g., query, load, extract, spreadsheetExtract), but there were 0: "
If I look at the docs at https://cloud.google.com/bigquery/docs/reference/v2/jobs#configuration.load it seems that the sourceUris[] is required but the example doesn't include it. My data is not in the google cloud storage but in the request. Why doesn't it detect my load details?
Thanks for any help. //Leif


Based on the message - looks like you are missing below
Content-Type: multipart/related; boundary="foo_bar_baz"


You were right. I had an extra space in my header for Content-Type, I saw it using fiddler and it looked like this:
`Content-Type : multipart/related; boundary="foo_bar_baz"
Thanks for your support.

Related

PhpStorm/WebStorm REST client editor: Using variables in JSON

I cannot use variables from http-client.env.json within the body as the following
POST {{host}}/{{locale}}/foo
Accept: application/json
Content-Type: application/json
{
"email": {{email}},
"password": {{password}}
}
so the string interpolation {{email}} is not being recognized for some reason as if it's not supported yet, and whenever I'm changing my code to hardly coded values as the following snippet things will be just fine!
POST {{host}}/{{locale}}/foo
Accept: application/json
Content-Type: application/json
{
"email": "info#example.com",
"password": "saynomore"
}
And obviously extracting the body into its own external JSON file is not an option.

Check documentation
https://www.jetbrains.com/help/pycharm/http-client-in-product-code-editor.html
https://www.jetbrains.com/help/pycharm/exploring-http-syntax.html
Preferable option json.env file, you can autogenerate it https://www.jetbrains.com/help/pycharm/exploring-http-syntax.html#c3cb9d37

HTTP 406 not acceptable response header fiddler

I am using fiddler 4 as a frond-end service for elasticsearch.
I'm trying to store a document into index using API following this.
Here's my sample json as request body:
{
"fruit": "Apple",
"size": "Large",
"color": "Red"
}
API : http://localhost:9200/food/fruit
method :POST HTTP/1.1
Here's the response:

As mentioned in their doc., "Starting from Elasticsearch 6.0, all REST requests that include a body must also provide the correct content-type for that body." adding content-type: application/json in the request solved the issue.

How to implement REST API Request URL/Headers/Body into https or code

I am working with Sabre's API (https://developer.sabre.com/io-docs). The documentation could leave a lot to be desired, especially for a noob. Sabre is giving me the following details to call a response that shows rental car availability. However I have no clue - do I take this data and load a HTTPS/URL? Do I write this into python code and execute it? Please help!
Request URI
https://api.test.sabre.com/v2.4.0/shop/cars
Request Headers
Authorization: Bearer T1RLAQI5V/zSp6o3WmZG10MSbQkPEkUShBCE+Bj0dkKsBv3/Rpv3u5ckAACgYUiSgZpk607+Y43m/veg1TSblwgOZPOpduj+Fa27070vBZfaWKmXT5od66o3K1C/N6Hu9KKniZC2fuplXuMkATtUEprzy8eud+Mz9e/0WVcqeQrnu3Z9Y/79WNDW3JGyxk8Gb05h5rdjavFPBHJaJE/ROPqw7fIsGZyj56EVEyP4BmcsWj3BSJkBEjBnon82Re5IPH94ntmX/HpiI/7V+g**
X-Originating-Ip: 98.116.41.181
Content-Type: application/json
Request Body
{
"OTA_VehAvailRateRQ": {
"VehAvailRQCore": {
"QueryType": "Shop",
"VehRentalCore": {
"PickUpDateTime": "04-08T09:00",
"ReturnDateTime": "04-09T11:00",
"PickUpLocation": {
"LocationCode": "DFW"
}
}
}
}
}

grails controller - POST json request

I have very simple controller with the following method:
def createNewWidgetVersion() {
println request.JSON
render status: OK
}
I'm sending the following request:
POST /CMSAdmin/widgetVersion/createNewWidgetVersion HTTP/1.1 Host:
localhost:8080
Content-Type: application/json
Cache-Control: no-cache
{ "htmlText":"33", "javaScript":
"document.getElementById('ffsdfdsfsfsfsd')", "comments":"SUCCESS",
"widgetId": "1", "type": "LIVE" }
and when debugging, IDE shows that request.JSON is empty. Controller doesn't extend RestfullController but I think it doesn't have to in this particular siutation. Could you please give a hint what I'm doing wrong? Spent few hours dealing with this. Thanks!
UPD1
I added the following to UrlMappings.groovy:
"/widgetVersion/"(controller: "widgetVersion", parseRequest:true){
action = [POST:'createNewWidgetVersion']
}
is there any way to avoid this? Thanks!

REST HATEOAS: How to determine and set media-type while browsing links?

I was going through what was described as an example of a good REST API.
A GET was sent on the base URI and with a media-type that was already known to the client somehow (which is fine, as per REST principles).
To server:
GET /
Host: xrgy.cloud.sun.com
Authorization: Basic xxxxxxxxxxxxxxxxxxx
Accept: application/vnd.com.sun.cloud.Cloud+json
X-Compute-Client-Specification-Version: 0.1
From Server:
HTTP/1.1 200 OK
Content-Type: application/vnd.com.sun.cloud.Cloud+json
Content-Length: nnn
{
"implementation_version": "597",
"vdcs": [
{
"name": "XRGY Virtual Data Center",
"uri": "/vdc"
}
{
"name": "R&D sandbox"
"uri": "/sandbox"
}
],
"uri": "http://xrgy.cloud.sun.com/",
"specification_version": [
"0.5"
]
}
But what I got stuck in was how the client set the media-type for the subsequent request.
I understand that the client got the URI for the next request from the previous response.
But where did it get the media-type from ? If it is prior knowledge to the client, then how do clients typically maintain such URI:media-type mappings ?
It seems I am definitely missing out on some basic knowledge here.
Here is the subsequent request sent with a media-type of : application/vnd.com.sun.cloud.Vdc+json !
To server:
GET /vdc
Host: xrgy.cloud.sun.com
Authorization: Basic xxxxxxxxxxxxxxxxxxx
Accept: application/vnd.com.sun.cloud.Vdc+json
X-Compute-Client-Specification-Version: 0.1
From server:
HTTP/1.1 200 OK
Content-Type: application/vnd.com.sun.cloud.Vdc+json
Content-Length: nnn
{
"name" : "XRGY Virtual Data Center",
"uri" : "http://xrgy.cloud.sun.com/vdc",
"vm_templates" : "http://cloud.sun.com/resources/template-cat.json",
"addresses" : [
{
"name": "144.34.100.199",
"uri": "/addresses/144.34.100.199",
"ip_address": "144.34.100.199"
}
],
"cluster" : {
"name" : "ROOT",
"uri" : "/vdc/",
"tags" : [ ],
"volumes" : [ ],
"clusters" : [
]
"tags" : [ ],
"controllers" : [
"start" : "/vdc/ops/start",
"stop" : "/vdc/ops/stop",
]
"vnets" : [
{
"name": "vnet1",
"uri": "/vnets/10.31.145.0",
"netmask": "255.255.255.0",
"network": "10.31.145.0"
}
],
"vms": [
{
* SNIPPED *
}
]
}
}
I have seen other examples where the media-type is also part of the links in the response, such as this following response and i can understand that.
201 Created
Content-Type: application/vnd.bank.org.transfer+xml;charset=UTF-8
<transfer xmlns="urn:org:bank:accounts">
<link rel="self"
href="http://bank.org/transfer/XTA8763"/>
<link rel="http://bank.org/rel/transfer/from"
type="application/vnd.bank.org.account+xml"
href="http://bank.org/account/AZA12093"/>
<link rel="http://bank.org/rel/transfer/to"
type="application/vnd.bank.org.account+xml"
href="http://bank.org/account/ADK31242"/>
<link rel="http://bank.org/rel/transfer/status"
type="application/vnd.bank.org.status+xml"
href="http://bank.org/check/XTA8763"/>
<id>transfer:XTA8763</id>
<amount currency="USD">100</amount>
<note>RESTing</note>
</transfer>

Simply put, yes, the client needs to have some a priori knowledge of the media types involved. Since the client actually sets the media types that it can consume. Since the client only understands "some" media types, if it sends a request with a media type that the application does not support, then the client is pretty much out of luck.
Since, in the the real world, we try to not have clients make blind calls to services returning payloads that they don't understand, the client will have some fore knowledge of the payloads involved, especially very specific types (vs plain/text or application/xml).
Finally, recall that the media type will effectively tell you the SYNTAX of the payloads but not how to interpret the payloads. Those semantics your client will have to know up front as well, so the burden of having an initial understanding of a media type is in fact not particularly a barrier to participation, it's just a fact of life.

Clients are responsible for telling servers what kind of response they understand/prefer. That is what the Accept header indicates. Servers respond with content that attempts to satisfy the client's request. The Content-Type header indicates what is actually returned. Ideally, the value of this header is the same as that of the Accept header.
See sections 14.1 and 14.17 in RFC 2616.
In your example, the client author probably is injecting knowledge into the client and this is not truly a 100% RESTful client.