Convert json data in cp1251 encoding - json

I work with data from web forms in cp1251;
I need to convert hash with data to JSON
When i used
my $href = {a => "дот"};
my $str = to_json($href, {utf8 => 0});
use Data::Dumper;
print Dumper($str); # will print something like this {"a": "\x{c4}\x{ee}\x{e8}"}
i need to make JSON string without converting.
In this case helps something like this:
my $str = encode('cp1251', decode('cp1251', to_json($href, {utf8 => 0}) ));
print Dumper($str); # will print {"a": "дот"}
or
my $str = to_json($href, {utf8 => 0});
utf8::downgrade($str);
print Dumper($str); # will print {"a": "дот"}
What other right options are there to solve the problem?

You can use the encode_json function from the JSON module to generate a JSON string from a data structure in a specified encoding. For example:
use JSON qw(encode_json);
my $href = {a => "дот"};
my $str = encode_json($href, {utf8 => 0, encode_check_method => 'sv'});
print Dumper($str); # will print {"a": "дот"}
The encode_check_method option specifies the method used to check for characters that cannot be encoded in the specified encoding. The value sv checks the SVf_UTF8 flag on the scalar, which means that it will only encode characters that are not flagged as UTF-8.
Alternatively, you can use the encode function from the Encode module to encode the JSON string after it has been generated:
use Encode qw(encode);
my $href = {a => "дот"};
my $json = to_json($href, {utf8 => 0});
my $str = encode('cp1251', $json);
print Dumper($str); # will print {"a": "дот"}
This will encode the JSON string in the cp1251 encoding after it has been generated.

Related

Not able to decode UTF-8 encoded json text with Cpanel::JSON::XS

I am trying to decode an UTF-8 encoded json string with Cpanel::JSON::XS:
use strict;
use warnings;
use open ':std', ':encoding(utf-8)';
use utf8;
use Cpanel::JSON::XS;
use Data::Dumper qw(Dumper);
my $str = '{ "title": "Outlining — How to outline" }';
my $hash = decode_json $str;
#my $hash = Cpanel::JSON::XS->new->utf8->decode_json( $str );
print Dumper($hash);
but this throws an exception at decode_json:
Wide character in subroutine entry
I also tried Cpanel::JSON::XS->new->utf8->decode_json( $str ) (see commented out line), but this gives another error:
malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "(end of string)")
What am I missing here?
decode_json expects UTF-8, but you are providing decoded text (a string of Unicode Code Points).
Use
use utf8;
use Encode qw( encode_utf8 );
my $json_utf8 = encode_utf8( '{ "title": "Outlining — How to outline" }' );
my $data = decode_json( $json_utf8 );
or
use utf8;
my $json_utf8 = do { no utf8; '{ "title": "Outlining — How to outline" }' };
my $data = decode_json( $json_utf8 );
or
use utf8;
my $json_ucp = '{ "title": "Outlining — How to outline" }';
my $data = Cpanel::JSON::XS->new->decode( $json_ucp ); # Implied: ->utf8(0)
(The middle one seems hackish to me. The first one might be used if you get data from multiple source, and the others provide it encoded.)

Perl LWP returns JSON output as string

I am using Perl LWP::UserAgent to get response from an API. Everything works good except one issue.
The API that i am using it returns response in JSON format. But I am getting it as string when i get the response through LWP module, Something like below.
$VAR1 = '
{"status":"success","data":[{"empid":"345232","customername":"Lee gates","dynamicid":"2342342332sd32423"},{"empid":"36.VLXP.013727..CBCL..","customername":"Lee subdirectories","dynamicid":"223f3423dsf23423423"}],"message":""}'
I did "print Dumper $response" to get the output.
One more thing, The challenge is that my client do not want to go with Perl module for JSON (use JSON::Parse 'parse_json';).
Any help would be appreciated!
You need to decode the JSON string into a Perl data structure. If your version of perl is 5.14+, JSON::PP is included in core, so nothing to install.
use warnings;
use strict;
use Data::Dumper;
use JSON::PP qw(decode_json);
my $json = '{"status":"success","data":[{"empid":"345232","customername":"Lee gates","dynamicid":"2342342332sd32423"},{"empid":"36.VLXP.013727..CBCL..","customername":"Lee subdirectories","dynamicid":"223f3423dsf23423423"}],"message":""}';
my $perl = decode_json $json;
print Dumper $perl;
Output:
$VAR1 = {
'status' => 'success',
'message' => '',
'data' => [
{
'dynamicid' => '2342342332sd32423',
'customername' => 'Lee gates',
'empid' => '345232'
},
{
'empid' => '36.VLXP.013727..CBCL..',
'customername' => 'Lee subdirectories',
'dynamicid' => '223f3423dsf23423423'
}
]
};

JSON encode/decode utf8 strings in Perl

I pass a utf8 encoded string from my command line into a Perl program:
> ./test.pl --string='ḷet ūs try ṭhiñgs'
which seems to recognize the string correctly:
use utf8;
GetOptions(
'string=s' => \$string,
) or die;
print Dumper($string);
print Dumper(utf8::is_utf8($string));
print Dumper(utf8::valid($string));
prints
$VAR1 = 'ḷet ūs try ṭhiñgs';
$VAR1 = '';
$VAR1 = 1;
When I store this string into a hash and call encode_json on it, the string seems to be again encoded whereas to_json seems to work (if I read the output correctly):
my %a = ( 'nāme' => $string ); # Note the Unicode character
print Dumper(\%a);
print Dumper(encode_json(\%a));
print Dumper(to_json(\%a));
prints
$VAR1 = {
"n\x{101}me" => 'ḷet ūs try ṭhiñgs'
};
$VAR1 = '{"nāme":"ḷet Å«s try á¹­hiñgs"}';
$VAR1 = "{\"n\x{101}me\":\"\x{e1}\x{b8}\x{b7}et \x{c5}\x{ab}s try \x{e1}\x{b9}\x{ad}hi\x{c3}\x{b1}gs\"}";
Turning this back into the original hash, however, doesn't seem to work with either methods and in both cases hash and string and broken:
print Dumper(decode_json(encode_json(\%a)));
print Dumper(from_json(to_json(\%a)));
prints
$VAR1 = {
"n\x{101}me" => "\x{e1}\x{b8}\x{b7}et \x{c5}\x{ab}s try \x{e1}\x{b9}\x{ad}hi\x{c3}\x{b1}gs"
};
$VAR1 = {
"n\x{101}me" => "\x{e1}\x{b8}\x{b7}et \x{c5}\x{ab}s try \x{e1}\x{b9}\x{ad}hi\x{c3}\x{b1}gs"
};
A hash lookup $a{'nāme'} now fails.
Question: How do I handle utf8 encoding and strings and JSON encode/decode correctly in Perl?
You need to decode your input:
use Encode;
my $string;
GetOptions('string=s' => \$string) or die;
$string = decode('UTF-8', $string);
Putting it all together, we get:
use strict;
use warnings;
use 5.012;
use utf8;
use Encode;
use Getopt::Long;
use JSON;
my $string;
GetOptions('string=s' => \$string) or die;
$string = decode('UTF-8', $string);
my %hash = ('nāme' => $string);
my $json = encode_json(\%hash);
my $href = decode_json($json);
binmode(STDOUT, ':encoding(utf8)');
say $href->{nāme};
Example:
$ perl test.pl --string='ḷet ūs try ṭhiñgs'
ḷet ūs try ṭhiñgs
Make sure your source file is actually encoded as UTF-8!

create a json string in perl

I am trying to create a json string in perl that outputs something like this:
{"d":{"success":false, "error":"key is required"}}
I have figured out how to do it without the "d" using this example:
my %rec_hash = ('a' => 1, 'b' => 2, 'c' => 3, 'd' => 4, 'e' => 5);
my $json = encode_json \%rec_hash;
print "$json\n";
but not sure what I'm suppose to do with the extra level
What you need is a hash reference:
use strict;
use warnings;
use JSON;
my $json = JSON->new;
my $data_to_json = {d=>{success=>JSON::false,error=>"key is required"}};
print $json->encode($data_to_json) . "\n";
The output is:
{"d":{"success":false,"error":"key is required"}}
Note the use of JSON::false to denote a value that the JSON module will correctly translate to false as a JSON value.

manipulating a JSON to correct format in perl

I have a JSON that prints
{"d":{"success":true,"drivers":[{"FIRST_NAME":"JOHN","LAST_NAME":"SMITH"},{"FIRST_NAME":"JANE","LAST_NAME":"DOE"}]}}
The names change depending on what was found in the database. I need to push this in this format for each result resturned in the JSON:
push(#$dummy_data, {'name' => 'testname', 'key' => 'somekey-1234'});
push(#$dummy_data, {'name' => 'testname2', 'key' => 'somekey-5678'});
So for this example it would be John Smith in place of testname and Jane for testname2
How would I do this so for each first and last name in the json gets pushed in the format above?
Let's try this new game
use strict; use warnings;
use JSON::XS;
use Data::Dumper;
# creating reference to a void ARRAY
my $dummy_data = [];
# creating $json string
my $json = '{"d":{"success":true,"drivers":[{"FIRST_NAME":"JOHN","LAST_NAME":"SMITH"},{"FIRST_NAME":"JANE","LAST_NAME":"DOE"}]}}';
# converting JSON -> Perl data structure
my $perl_hash = decode_json $json;
# feeding $dummy_data ARRAY ref with a HASH
push #$dummy_data, {
name => $perl_hash->{d}->{drivers}->[0]->{FIRST_NAME},
key => $perl_hash->{d}->{drivers}->[1]->{FIRST_NAME}
};
# print what we have finally
print Dumper $dummy_data;
Output
$VAR1 = [
{
'name' => 'JOHN',
'key' => 'JANE'
}
];