I'm trying to run GET method using LWP and CURL (unix command) to get the content in JSON format, I'm seeing the complete content only in CURL command but the LWP response is showing valid JSON but with empty values.
REQUEST
LWP
my $ua= LWP::UserAgent->new();
my $host='https://sample.com/iersiefhsiof';
$ua->default_header('Content-Type' => 'application/json');
$ua->default_header('Authorization' => 'XXXXXXXXXXXXXXxx');
$ua->default_header('Accept' => 'application/json');
my $response=$ua->get($host);
$res=$response->content();
print ($res);
CURL
curl -H "Accept: application/json" -H "Content-Type: application/json" -H 'Authorization:XXXXXXXXXXXXXXX' -ikL 'https://sample.com/iersiefhsiof'
RESPONSE
LWP
{
"id":"",
"state":"",
"category":"",
"severity":"",}'`
CURL
`{"id":"iersiefhsiof","state":"open","category":"App","severity":"minor",}'`
Please help me to understand what I'm missing in LWP request to get the complete response content.
It is fixed now !
As per Header.pm , I should use ":" for custom header field name , if I'm not using that then it is converting the first characters as Uppercase and hence the header is missed.
unless ($field =~ /^:/) {
$field =~ tr/_/-/ if $TRANSLATE_UNDERSCORE;
my $old = $field;
$field = lc $field;
unless($standard_case{$field} || $self->{'::std_case'}{$field}) {
# generate a %std_case entry for this field
**$old =~ s/\b(\w)/\u$1/g;**
$self->{'::std_case'}{$field} = $old;
}
}
Refer: https://github.com/libwww-perl/HTTP-Message/blob/master/lib/HTTP/Headers.pm
':x-inf-route-key'=>'GetID'
Thanks :)
Related
I have the below curl command which i am trying to convert to powershell, but i am not sure on how to handle the multi header inputs and the JSON input
Any help is highly appreciated
curl --location --request POST 'https://anypoint.studio.com/cloudhub/api/v2/applications' \
--header 'X-ANYPNT-ENV-ID: 4a96abfd4f5237cf1b64' \
--header 'X-ANYPNT-ORG-ID: bc0d-3b9fd79234ad' \
--header 'Authorization: Bearer 3esede-a44b-29ab8841b508' \
--form 'file=#"/C:/Users/pgi/Downloads/eafdc-flow-proxy-v1.2.jar"' \
--form 'appInfoJson="{
\"domain\": \"testproxy-test\",
\"Version\": {
\"version\": \"4.4.0\"
},
\"properties\": {
\"platform.client_id\": \"8f95qw3sa4b679aaa699cf0f5c6b6\",
\"secure.key\": \"sYf%NJ7F^y&3lNRH*D$#\",
\"env\": \"dev\",
\"platform.client_secret\": \"acF0JHFUEFH8829a481D9c37EF364be7a\"
},
\"propertiesOptions\": {
\"secure.key\": {
\"secure\": true
},
\"anypoint.platform.client_secret\": {
\"secure\": true
}
},
\"region\": \"us-west-1\",
\"monitoringEnabled\": true,
\"monitoringAutoRestart\": true,
\"workers\": {
\"amount\": 1,
\"type\": {
\"name\": \"Micro\",
\"weight\": 0.1,
\"cpu\": \"0.1 vCores\",
\"memory\": \"500 MB memory\"
}
},
\"loggingNgEnabled\": true,
\"persistentQueues\": true
}"' \
--form 'autoStart="true"'
You can replace curl with Invoke-WebRequest and parse the JSON with ConvertFrom-Json.
$r = Invoke-WebRequest -Uri $uri -Method Post -Body $body
$j = ConvertFrom-Json $r.Content
I've left out setting the value of $uri and $body. I hope this is enough for you to get started with.
Update
First response was written too quickly. You probably need to pass header and body.
Like curl, Invoke-Webrequest is powerful and complex. You can find online documentation here:
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-webrequest?view=powershell-7.2&WT.mc_id=ps-gethelp
You'll need to use the Headers and Body Parameters:
-Headers <System.Collections.IDictionary>
Specifies the headers of the web request. Enter a hash table or dictionary.
To set UserAgent headers, use the UserAgent parameter. You can't use this parameter to specify User-Agent or cookie headers.
Content related headers, such as Content-Type is overridden when a MultipartFormDataContent object is supplied for Body .
-Body <System.Object>
Specifies the body of the request. The body is the content of the request that follows the headers. You can also pipe a body value to Invoke-WebRequest.
The Body parameter can be used to specify a list of query parameters or specify the content of the response.
When the input is a GET request and the body is an IDictionary (typically, a hash table), the body is added to the URI as query
parameters. For other request types (such as
POST), the body is set as the value of the request body in the standard name=value format.
The Body parameter may also accept a System.Net.Http.MultipartFormDataContent object. This facilitates multipart/form-data requests. When a MultipartFormDataContent object is supplied for Body , any Content related headers supplied to the ContentType , Headers , or WebSession parameters is overridden by the Content headers of the MultipartFormDataContent object. This feature was added in PowerShell 6.0.0.
Looks like you'll have to create a MultipartFormDataContent object. Note this was added in PowerShell Version 6, so you can't use the Windows built-in default version 5. If you haven't already, you'll probably need to install the latest version of Version 7. (I use version 7 and it's great.)
So for headers you'd create something like:
$headers = [ordered]#{
'X-ANYPNT-ENV-ID' = '4a96abfd4f5237cf1b64'
'X-ANYPNT-ORG-ID' = 'bc0d-3b9fd79234ad'
'Authorization' = 'Bearer 3esede-a44b-29ab8841b508'
}
Then make a call like:
$r = Invoke-WebRequest -Uri $uri -Method Post `
-Headers $headers -Body $myMultipartFormDataContent
I haven't used MultipartFormDataContent so I'll leave that up to you. I hope this provides you with a good starting point.
Just curious to know the difference between below three executions, If I use a variable to hold the json data and pass the same to curl command, json msg become corrupt and server returns incorrect result, What is the best way to pass json from a variable while invoking curl command ? Trying to make the last line in the snippet work.
my $tsFormatted = '2021-06-10';
my $metaLoad = '{"validDate": "'.$tsFormatted.'"}';
my $curl_out_1 = `curl -s -X POST -H 'Content-Type: application/json' $mw_meta_load_api --key $mw_access_key --cert $mw_access_cert --data '{"validDate": "2021-06-10"}'`;
my $curl_out_2 = `curl -s -X POST -H 'Content-Type: application/json' $mw_meta_load_api --key $mw_access_key --cert $mw_access_cert --data '{"validDate": "'.$tsFormatted.'"}'`;
my $curl_out_3 = `curl -s -X POST -H 'Content-Type: application/json' $mw_meta_load_api --key $mw_access_key --cert $mw_access_cert --data $metaLoad`;
Backticks interpolate as double-quoted strings, so we can easily view the difference by using double-quoted string literals instead of backticks.
my $tsFormatted = '2021-06-10';
my $metaLoad = '{"validDate": "'.$tsFormatted.'"}';
# qq`...`, qq"..." and "..." are all the same thing.
say qq`... --data '{"validDate": "2021-06-10"}'`;
say qq`... --data '{"validDate": "'.$tsFormatted.'"}'`;
say qq`... --data $metaLoad`;
Output:
... --data '{"validDate": "2021-06-10"}'
... --data '{"validDate": "'.2021-06-10.'"}'
... --data {"validDate": "2021-06-10"}
The contents of string literals (including backticks) aren't treated as Perl code.
Of those three, only the first is correct.
The first passes the following two arguments to curl:
--data
{"validDate": "2021-06-10"}
The second passes the following two arguments to curl:
--data
{"validDate": ".2021-06-10."}
The third passes the following three arguments to curl:
--data
{validDate:
2021-06-10}
What you want:
use Cpanel::JSON::XS qw( encode_json );
use String::ShellQuote qw( shell_quote );
my $ts_formatted = '2021-06-10';
my $data = { validDate => $ts_formatted };
my $data_formatted = encode_json($data);
my $cmd = shell_quote(
curl => (
$mw_meta_load_api,
'-s',
-X => 'POST',
-H => 'Content-Type: application/json',
'--key' => $mw_access_key,
'--cert' => $mw_access_cert,
'--data' => $data_formatted,
)
);
my $out = `$cmd`;
die("Can't execute shell: $!\n") if $? == -1;
die("curl killed by signal ".( $? & 0x7F )."\n") if $? & 0x7F;
die("curl exited with error ".( $? >> 8 )."\n") if $? >> 8;
Also, have you considered using Net::Curl::Easy instead of shelling out?
I am currently trying to automate new user creation in our Zendesk ticketing system using Powershell and Curl. The problem I am running into is that the curl json body is enclosed by single quotes and I need to reference a variable inside that body. Here is what I have:
$Firstname = "Test"
$Lastname = "User"
$email= 'user#test.org'
curl.exe https://mydomain.zendesk.com/api/v2/users.json -H "Content-Type: application/json" -X POST -d '{\"user\": {\"name\": \"$Firstname $Lastname\", \"email\": \"$email\"}}' -v -u myuser:mypass
This works fine if I type in regular text values inside the json, but how can I get it to recognize the variables $Firstname, $Lastname and $email?
Try the following:
$Firstname = "Test"
$Lastname = "User"
$email= 'user#test.org'
$json=#"
{\"user\": {\"name\": \"$Firstname $Lastname\", \"email\": \"$email\"}}
"#
curl.exe https://mydomain.zendesk.com/api/v2/users.json -d $json -H 'Content-Type: application/json' -X POST -v -u myuser:mypass
Using a double-quoted here-string #"<newline>...<newline>"# makes specifying embedded " instances easy (no escaping for the sake of PowerShell's own syntax required), while still expanding variable references - see the docs online or Get-Help about_Quoting_Rules.
You're clearly aware of the - unfortunate - additional need to \-escape the " instances, but just to explain why that is needed:
When passing arguments to an external program, PowerShell, after its own parsing and interpolation, conditionally wraps the resulting arguments in double quotes when concatenating them to form the argument list (a single string) to pass to the external utility. Unfortunately, this can result in embedded " instances getting discarded, and the only way to preserve them reliably is to \-escape them - see this answer for more information.
If you wanted to do it inline with a regular double-quoted string, you'd have to escape the " instances for PowerShell too (as `"), resulting in the awkward combination \`":
"{\`"user\`": {\`"name\`": \`"$Firstname $Lastname\`", \`"email\`": \`"$email\`"}}"
Afterthought:
Ryan himself points out in a comment that using a hashtable to construct the data and then converting it to JSON with ConvertTo-Json and feeding it to curl via stdin is an alternative that avoids the quoting headaches:
# Create data as PS hashtable literal.
$data = #{ user = #{ name = "$Firstname $Lastname"; email = "$adUsername#mydomain.org" } }
# Convert to JSON with ConvertTo-Json and pipe to `curl` via *stdin* (-d '#-')
$data | ConvertTo-Json -Compress | curl.exe mydomain.zendesk.com/api/v2/users.json -d '#-' -H "Content-Type: application/json" -X POST -v -u myuser:mypass
I think we can use here-string as json body for Invoke-RestMethod as below
$bufferTime = 5
$requestBody = #"
{
"size": 0,
"aggs": {
"last_x_min": {
"filter": {
"range": {
"#timestamp": {
"gte": "now-$($bufferTime)m",
"lte": "now"
}
}
},
"aggs": {
"value_agg": {
"avg": {
"field": "time-taken"
}
}
}
}
}
}
"#
$esResponse = Invoke-RestMethod -URI http://locahost:9200 -TimeoutSec 15 -Method Post -ContentType 'application/json' -Body $requestBody
This is the script I use to query Elasticsearch. No need to escape double quotes.
I use this curl command:
curl -X POST -H "Content-Type: application/json" http://localhost:8081/creditcard -d '{"credit-card":"1234-5678-9101-1121"}'
In my js file, I have this code block to get the credit-card's value:
request.on('data', function(data) {
var cc = 'credit-card';
var a = JSON.parse(data.toString());
console.log(a[cc]);
}
For this I get:
undefined:1
'{credit-card:1234-5678-9101-1121}'
^
SyntaxError: Unexpected token '
at Object.parse (native)
at IncomingMessage.<anonymous> (<path>\ccserver.js:32:34)
at IncomingMessage.emit (events.js:107:17)
at IncomingMessage.Readable.read (_stream_readable.js:373:10)
at flow (_stream_readable.js:750:26)
at resume_ (_stream_readable.js:730:3)
at _stream_readable.js:717:7
at process._tickCallback (node.js:355:11)
So I tried to use JSON.stringify as followed:
request.on('data', function(data) {
var cc = 'credit-card';
var a = JSON.parse(JSON.stringify(data.toString()));
console.log(a[cc]);
}
But this is what I get:
undefined
undefined
However, when I try to parse a hard-coded json string, it goes ok:
var jsonString = '{"credit-card":"1234-5678-9101-1121"}';
var a = JSON.parse(jsonString);
console.log(a[cc]);
Result:
1234-5678-9101-1121
What is the correct way to do get the data out of this json?
Please advise
Thanks
Try reading from absolute path
curl -X POST
-H 'Content-Type:application/json'
-H 'Accept: application/json'
--data-binary #/full/path/to/test.json
http://server:port/xyz/abc/blah -v -s
Well, you already have String so all you need to convert it to javascript variable and get using .notation. Suggest to use firebug to see what is in variable.
obj = JSON.parse(json);
obj.cc or obj.cc[0]
should give you what you want.
I have inherited a controller.
When a post request is made, with a well structured JSON document everything is fine.
When the JSON contains a space in a feld, 404 is returned.
However, when the same request is made from mozilla restclient extension everything works.
The CURL request specifically is:
curl --include \
--request POST \
--header "Content-Type: application/json" \
--header "Accept: application/json" \
--data-binary "{
"planCode" : "My Test App-standard"
}" \
"https://localhost/signupApi/v2/signup"
URLMapping:
"/signupApi/v2/$action"{
parseRequest = true // parse json, and assign to params
controller = "signupApiSignup"
}
So, why would a space in curl cause problems in the request body that grails receives?
Thanks
You are not quoting your strings there properly in your shell. Use ' for your parameters, if you plan to use "sensitive" chars like " there. Or use \" inside. Also curl can read a filename if you prefix it with # instead of the actual data.
Yet in this case maybe quoting with ' is easiest. E.g.:
...
--data-binary '{
"planCode" : "My Test App-standard"
}'
...