Perl json keys with spaces [duplicate] - json

This question already has answers here:
Which Perl module would you recommend for JSON manipulation?
(6 answers)
Closed 2 years ago.
how can i parse perl json object which has spaces in its keys
{
"abc" : [
"lmn" : {
"Ab Cd" : "Xy Zw",
"Ef Gh" : "Pq Rs",
}
]
}

By definition, one parses JSON using a JSON parser. There exists multiple JSON parsers on CPAN, including Cpanel::JSON::XS. It handles keys with spaces in them without issue, as should every other JSON parser.
Note that what you have isn't JSON. I'm assuming the errors are typos since you asked about JSON.

Spaces in a key will present no problems at all to any JSON parser.
There are, however, two problems in your JSON that will cause problems for any parser. Others have noted the extra comma after "Pq Rs", but you also have an array that contains a key/value pair (with the key "lnm") which needs to be inside an object.
Originally, I just removed the comma and ran this code:
#!/usr/bin/perl
use strict;
use warnings;
use feature 'say';
use Data::Dumper;
use JSON;
my $json = '{
"abc" : [
"lmn" : {
"Ab Cd" : "Xy Zw",
"Ef Gh" : "Pq Rs"
}
]
}';
my $data = decode_json($json);
say Dumper $data;
This gives an error:
, or ] expected while parsing array, at character offset 28 (before ": {\n "Ab C...")
I fixed it, by inserting { ... } around the lnm object.
#!/usr/bin/perl
use strict;
use warnings;
use feature 'say';
use Data::Dumper;
use JSON;
my $json = '{
"abc" : [ {
"lmn" : {
"Ab Cd" : "Xy Zw",
"Ef Gh" : "Pq Rs"
}
} ]
}';
my $data = decode_json($json);
say Dumper $data;
And then I got this output:
$VAR1 = {
'abc' => [
{
'lmn' => {
'Ab Cd' => 'Xy Zw',
'Ef Gh' => 'Pq Rs'
}
}
]
};
Which is, I think, what you are expecting.

Related

Perl LWP::UserAgent parse response JSON

I am using the LWP::UserAgent module to issue a GET request to one of our APIs.
#!/usr/bin/perl
use strict;
use warning;
use LWP::UserAgent;
use Data::Dumper;
my $ua = LWP::UserAgent->new;
my $request = $ua->get("http://example.com/foo", Authorization => "Bearer abc123", Accept => "application/json" );
print Dumper $request->content;
The request is successful. Dumper returns the following JSON.
$VAR1 = '{
"apiVersion": "v1",
"data": {
"ca-bundle.crt": "-----BEGIN CERTIFICATE-----abc123-----END CERTIFICATE-----\\n"
},
"kind": "ConfigMap",
"metadata": {
"creationTimestamp": "2021-07-16T17:13:01Z",
"labels": {
"auth.openshift.io/managed-certificate-type": "ca-bundle"
},
"managedFields": [
{
"apiVersion": "v1",
"fieldsType": "FieldsV1",
"fieldsV1": {
"f:data": {
".": {},
"f:ca-bundle.crt": {}
},
"f:metadata": {
"f:labels": {
".": {},
"f:auth.openshift.io/managed-certificate-type": {}
}
}
},
"manager": "cluster-kube-apiserver-operator",
"operation": "Update",
"time": "2021-09-14T17:07:39Z"
}
],
"name": "kube-control-plane-signer-ca",
"namespace": "openshift-kube-apiserver-operator",
"resourceVersion": "65461225",
"selfLink": "/api/v1/namespaces/openshift-kube-apiserver-operator/configmaps/kube-control-plane-signer-ca",
"uid": "f9aea067-1234-5678-9101-9d4073f5ae53"
}
}';
Let's say I want to print the value of the apiVersion key, which should print v1.
print "API Version = $request->content->{'apiVersion'} \n";
The following is being printed. I am not sure how to print the value v1. Since HTTP::Response is included in the output, I suspect I might have to use the HTTP::Response module?
API Version = HTTP::Response=HASH(0x2dffe80)->content->{'apiVersion'}
Perl doesn't expand subroutine calls in a double-quoted string.
print "API Version = $request->content->{'apiVersion'} \n";
In this line of code, content() is a subroutine call. So Perl sees this as:
print "API Version = $request" . "->content->{'apiVersion'} \n";
And if you try to print most Perl objects, you'll get the hash reference along with the name of the class - hence HTTP::Response=HASH(0x2dffe80).
You might think that you just need to break up your print() statement like this:
print 'API Version = ', $request->content->{'apiVersion'}, "\n";
But that's not going to work either. $request->content doesn't return a Perl data structure, it returns a JSON-encoded string. You need to decode it into a data structure before you can access the individual elements.
use JSON;
print 'API Version = ', decode_json($request->content)->{'apiVersion'}, "\n";
But it might be cleaner to do the decoding outside of the print() statement.
use JSON;
my $data = decode_json($request->content);
In which case you can go back to something more like your original code:
print "API Version = $data->{'apiVersion'} \n";
The JSON content must be decoded first. There are several modules for that, like JSON:
use JSON;
# ...
my $href = decode_json $request->content;
And then use it like a normal hash reference: $href->{apiVersion}

How to extract certain data using Perl from a file?

I have data that needs to be extracted from a file, the lines I need for the moment are name,location and host. This is example of the extract. How would I go about getting these lines into a separate file? I have the Original file and the new file i want to create as the input/output file, there are thousands of devices contained within the output file and they are all the same formatting as in my example.
#!/usr/bin/perl
use strict;
use warnings;
use POSIX qw(strftime);
#names of files to be input output
my $inputfile = "/home/nmis/nmis_export.csv";
my $outputfile = "/home/nmis/nmis_data.csv";
open(INPUT,'<',$inputfile) or die $!;
open(OUTPUT, '>',$outputfile) or die $!;
my #data = <INPUT>;
close INPUT;
my $line="";
foreach $line (#data)
{
======Sample Extract=======
**"group" : "NMIS8",
"host" : "1.2.3.4",
"location" : "WATERLOO",
"max_msg_size" : 1472,
"max_repetitions" : 0,
"model" : "automatic",
"netType" : "lan",
"ping" : 1,
"polling_policy" : "default",
"port" : 161,
"rancid" : 0,
"roleType" : "access",
"serviceStatus" : "Production",
"services" : null,
"threshold" : 1,
"timezone" : 0,
"version" : "snmpv2c",
"webserver" : 0
},
"lastupdate" : 1616690858,
"name" : "test",
"overrides" : {}
},
{
"activated" : {
"NMIS" : 1
},
"addresses" : [],
"aliases" : [],
"configuration" : {
"Stratum" : 3,
"active" : 1,
"businessService" : "",
"calls" : 0,
"cbqos" : "none",
"collect" : 0,
"community" : "public",
"depend" : [
"N/A"
],
"group" : "NMIS8",
"host" : "1.2.3.5",
"location" : "WATERLOO",
"max_msg_size" : 1472,
"max_repetitions" : 0,
"model" : "automatic",
"netType" : "lan",
"ping" : 1,
"polling_policy" : "default",
"port" : 161,
"rancid" : 0,
"roleType" : "access",
"serviceStatus" : "Production",
"services" : null,
"threshold" : 1,
"timezone" : 0,
"version" : "snmpv2c",
"webserver" : 0
},
"lastupdate" : 1616690858,
"name" : "test2",
"overrides" : {}
},**
I would use jq for this not Perl. You just need to query a JSON document. That's what jq is for. You can see an example here
The jq query I created is this one,
.[] | {name: .name, group: .configuration.group, location: .configuration.location}
This breaks down into
.[] # iterate over the array
| # create a filter to send it to
{ # that produces an object with the bellow key/values
.name,
group: .configuration.group,
location: .configuration.location
}
It provides an output like this,
{
"name": "test2",
"group": "NMIS8",
"location": "WATERLOO"
}
{
"name": "test2",
"group": "NMIS8",
"location": "WATERLOO"
}
You can use this to generate a csv
jq -R '.[] | [.name, .configuration.group, .configuration.location] | #csv' ./file.json
Or this to generate a csv with a header,
jq -R '["name","group","location"], (.[] | [.name, .configuration.group, .configuration.location]) | #csv' ./file.json
You can use the JSON distribution for this. Read the entire file in one fell swoop to put the entire JSON string into a scalar (as opposed to putting it into an array and iterating over it), then simply decode the string into a Perl data structure:
use warnings;
use strict;
use JSON;
my $file = 'file.json';
my $json_string;
{
local $/; # Locally reset line endings to nothing
open my $fh, '<', $file or die "Can't open file $file!: $!";
$json_string = <$fh>; # Slurp in the entire file
}
my $perl_data_structure = decode_json $json_string;
As what you have there is JSON, you should parse it with a JSON parser. JSON::PP is part of the standard Perl distribution. If you want something faster, you could install something else from CPAN.
Update: I included a link to JSON::PP in my answer. Did you follow that link? If you did, you would have seen the documentation for the module. That has more information about how to use the module than I could include in an answer on SO.
But it's possible that you need a little more high-level information. The documentation says this:
JSON::PP is a pure perl JSON decoder/encoder
But perhaps you don't know what that means. So here's a primer.
JSON is a text format for storing complex data structures. The format was initially used in Javascript (the acronym stands for "JavaScript Object Notation") but it is now a standard that is used across pretty much all programming languages.
You rarely want to actually deal with JSON in a program. A JSON document is just text and manipulating that would require some complex regular expressions. When dealing with JSON, the usual approach is to "decode" the JSON into a data structure inside your program. You can then manipulate the data structure however you want before (optionally) "encoding" the data structure back into JSON so you can write it to an output file (in your case, you don't need to do that as you want your output as CSV).
So there are pretty much only two things that a Perl JSON library needs to do:
Take some JSON text and decode it into a Perl data structure
Take a Perl data structure and encode it into JSON text
If you look at the JSON::PP documentation you'll see that it contains two functions, encode_json() and decode_json() which do what I describe above. There's also an OO interface, but let's not overcomplicate things too quickly.
So your program now needs to have the following steps:
Read the JSON from the input file
Decode the JSON into a Perl data structure
Walk the Perl data structure to extract the items that you need
Write the required items into your output file (for which Text::CSV will be useful
Having said all that, it really does seem to me that the jq solution suggested by user157251 is a much better idea.

looping through json in perl

I'm trying to grab some information out of a json export from Ping. My rusty Perl skills are failing me as I'm getting lost in the weeds with the dereferencing. Rather than bang my head against the wall some more I thought I'd post a question since all the google searches are leading here.
My understanding is that decode_json converts items into an array of hashes and each hash has strings and some other arrays of hashes as contents. This seems to bear out when attempting to get to an individual string value but only if I manually specify a specific array element. I can't figure out how to loop through the items.
The JSON comes back like this:
{
"items":[
{
#lots of values here are some examples
"type": "SP",
"contactInfo": {
"company": "Acme",
"email": "john.doe#acme.com"
}
]
}
I had no problems getting to actual values
#!/usr/bin/perl
use JSON;
use Data::Dumper;
use strict;
use warnings;
use LWP::Simple;
my $json;
{
local $/; #Enable 'slurp' mode
open my $fh, "<", "idp.json";
$json = <$fh>;
close $fh;
}
my $data = decode_json($json);
#array print $data->{'items'};
#hash print $data->{'items'}->[0];
#print $data->{'items'}->[0]->{'type'};
But, I can't figure out how to iterate through the array of items. I've tried for and foreach and various combinations of dereferencing, and it keeps telling me that the value I'm looping thru is still an array. If $data->{'items'} is an array, then presumably I should be able to do some variation of
foreach my $item ($data->{'items'})
or
my #items = $data->{'items'};
for (#items)
{
# stuff
}
But, I keep getting arrays back and I have to add in the ->[0] to get to a specific value.
$data->{'items'} is a reference to an array (of hash references). You need to dereference it, with #{ }:
use JSON;
use strict;
use warnings;
my $json;
{
local $/; #Enable 'slurp' mode
$json = <DATA>;
}
my $data = decode_json($json);
for my $item (#{ $data->{items} }) {
print "$item->{type}\n";
}
__DATA__
{
"items":[
{
"type": "SP",
"contactInfo": {
"company": "Acme",
"email": "john.doe#acme.com"
}
}
]
}
Output:
SP

perl json encode converting numbers to strings [duplicate]

This question already has answers here:
Converting array of numbers to json in perl, getting array of strings
(3 answers)
Closed 8 years ago.
I am trying to encode a perl nested hash and send it to some web application.
Somehow the json encoder converts the numbers or floats to strings.
The web application sees the data as strings and can't plot the chart. I can add code in the web application to convert them back to numbers, but I am looking for better solution of not having the numbers as strings in the first place.
Here is the code:
use strict;
use warnings;
use CGI qw/param/;
use JSON::XS;
my $json_obj = JSON::XS->new->allow_nonref;
## Build some Perl data
my %perl_data;
$perl_data{'numbers'}{'nested'} = [qw/1 -2 4 2 5 6/] ;
$perl_data{'mix'}{'AnotherLevel'} = [qw/null "Temp" 4 2 5 6/] ;
print "Content-type: text/html\n\n";
print $json_obj->pretty->encode(\%perl_data);
Here is the output where everything is just stringified:
Content-type: text/html
{
"numbers" : {
"nested" : [
"1",
"-2",
"4",
"2",
"5",
"6"
]
},
"mix" : {
"AnotherLevel" : [
"null",
"\"Temp\"",
"4",
"2",
"5",
"6"
]
}
}
In the above code, I even tried the following, but to no avail.
use JSON;
my $json_obj = JSON;
Any help is greatly appreciated.
The JSON::XS documentation actually has a good section which describes how Perl data structures are serialized as JSON. In particular, it says for scalars:
JSON::XS will encode undefined scalars as JSON null values, scalars that have last
been used in a string context before encoding as JSON strings, and anything else
as number values.
When you define your array of numbers using qw, you are using them in a string context. (qw means "quote word" and is generally used to save some typing when defining a list of words.)
Note also that null in JSON is represented by the undef value in Perl. When you say qw/null/, you're just creating the literal string 'null'.
So you have two options.
Define your array like this:
$perl_data{'numbers'}{'nested'} = [1, -2, 4, 2, 5, 6] ;
$perl_data{'mix'}{'AnotherLevel'} = [undef, "Temp", 4, 2, 5, 6] ;
Or, force-numify all your numbers by adding zero to them before you serialize. E.g.
$perl_data{'numbers'}{'nested'} = [ map { $_ + 0 } qw/1 -2 4 2 5 6/ ];
Don't initialize them as strings, and you won't have a problem:
use strict;
use warnings;
use CGI qw/param/;
use JSON::XS;
my $json_obj = JSON::XS->new->allow_nonref;
## Build some Perl data
my %perl_data = (
'numbers' => {'nested' => [1, -2, 4, 2, 5, 6]},
'mix' => {'AnotherLevel' => [qw/null "Temp"/, 4, 2, 5, 6]},
);
print "Content-type: text/html\n\n";
print $json_obj->pretty->encode(\%perl_data);
Outputs:
Content-type: text/html
{
"numbers" : {
"nested" : [
1,
-2,
4,
2,
5,
6
]
},
"mix" : {
"AnotherLevel" : [
"null",
"\"Temp\"",
4,
2,
5,
6
]
}
}

Parsing simple JSON array in Perl [duplicate]

This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Parse perl array
I'm trying to edit an old perl script and I'm a complete beginner.
The request from the server returns as:
$result = {
"data": {
"translations": [
{
"translatedText": "Halloween"
}
]
}
}
How can I parse this JSON string to grab:
$result = "Halloween"
Thanks.
#!/usr/bin/perl
use strict;
use warnings;
use 5.010;
use JSON;
my $json = '{
"data": {
"translations": [
{
"translatedText": "Halloween"
}
]
}
}';
my $data = decode_json($json);
say $data->{data}{translations}[0]{translatedText};
There are uncountable JSON parsing modules available; the most standard-like one is JSON.
You could take the easy way and do it like this (didn't check if this compiles):
my $translatedText = ""
if ($result =~ /"translatedText": "(.+?)"/)
{
$translatedText = $1;
}
if you want to parse for more items and different json strings then you should use some perl module (there are a few avaialble)