True stringify of DateTime - json

According to:
http://search.cpan.org/~drolsky/DateTime-1.43/lib/DateTime.pm#Formatters_And_Stringification
the following will work:
use DateTime;
$dt = DateTime->new( ... );
print $dt; # as string
and it does.
However, when using the JSON module to embed the created $dt string as a json-encoded data structure as follows:
use JSON;
my $json = JSON::encode_json( { dt => $dt } );
The JSON module is throwing the following exception:
encountered object '2017-08-02', but neither allow_blessed nor convert_blessed settings are enabled at ...
I can work around the issue as follows:
my $json = JSON::encode_json( { dt => substr($dt,0) } );
but wonder if there's a cleaner way to get a "true" string out of DateTime?
I've looked through the DateTime docs but I don't see an explicit stringify() to create the true string as required by JSON.
EDIT: In the constructor, I specify a desired output format via 'formatter'. I'd like to get a string using the already-specified format, and not specify the format again (which would be against DRY principle...)
EDIT2: Interpolating the string is a good solution if just $dt, but doesn't work if $dt is actually a moose attribute (i.e. "$person->birth_d").

You can use "$dt" or "".$o->dt. Like your solution, it's short for
$dt->formatter ? $dt->formatter->format_datetime($dt) : $dt->iso8601
If you want a specific format, there's $dt->strftime, $dt->iso8601, or you could use a DateTime::Format:: module.

You can explicitly interpolate the object in a string ("$dt") or call the method that's actually getting called ($dt->iso8601).
use strict;
use warnings;
use 5.010;
use DateTime;
my $dt = DateTime->now;
say $dt;
say "$dt";
say $dt->iso8601;
Output:
2017-08-02T19:54:22
2017-08-02T19:54:22
2017-08-02T19:54:22

Related

Why is it necessary to quote Perl version string for JSON encoding?

For some Perl diagnostic tests, I'm recording assorted bits of information formatted as JSON using JSON::MaybeXS.
I get an error when I want to record the current Perl version, which I obtain from the special variable $^V.
As the minimal demonstration script shows, the error occurs unless I quote $^V as "$^V".
json_perl_version_test.pl
#!/usr/bin/env perl
use strict;
use warnings;
use v5.18;
use JSON::MaybeXS;
say "Running Perl version $^V";
my $item = 'Wut?';
my %hash1 = (
something => $item,
v_unquoted => $^V
);
eval { say say 'Hash1: ', encode_json \%hash1 };
say "Oops - JSON encode error: $#" if $#;
my %hash2 = (
something => $item,
v_quoted => "$^V"
);
say 'Hash2: ', encode_json \%hash2;
# Running Perl version v5.34.0
# Oops - JSON encode error: encountered object 'v5.34.0',
# but neither allow_blessed, convert_blessed nor allow_tags
# settings are enabled (or TO_JSON/FREEZE method missing) at
# /Users/bw/Documents/Dev/tests/json_perl_version_test.pl line 17.
# Hash2: {"something":"Wut?","v_quoted":"v5.34.0"}
Note that it wasn't necessary to quote $item.
The error message refers to some ways to handle other cases, but seemingly not including canonical Perl version dotted-decimal strings. I've looked through the main Perl JSON modules (recent versions of JSON::MaybeXS, JSON, and Cpanel::JSON::XS), but can't find anything referring to $^V or dotted-decimal strings. Also don't find a relevant question on SO :(.
Perhaps I'm missing something? Or am I stuck with needing to quote $^V?
Reasons?
Thanks,
The $^V variable is really an object
The revision, version, and subversion of the Perl interpreter, represented as a version object.
An object cannot be stored in JSON just so. Quoting it stringifies it.
It is possible to make JSON::XS (and its Cpanel::) take a blessed reference but it involves more work. See Object Serialization. The cleanest complete solution is with convert_blessed, when the encode method will look for a TO_JSON method (in the class of the object that is to be added to JSON), which would return a JSON-ready string.
Alas, there is no such a thing for the version (nor for a few other classes I tried, like DateTime). One can add that method to the class† but just thinking of it makes quotes look nice.
Another way is to get explicit in making the version object stringify
my $json = JSON::XS->new->encode( { ver => $^V->stringify } )
This is yet more elaborate but at least now it's clear what the matter is, without magic quotes.
Or just quote it and add a comment.
† By "monkey-patching" it, for example
add the sub, with a fully qualified name
perl -Mstrict -wE'use JSON::XS; say $^V;
sub version::TO_JSON { return $_[0]->stringify };
my $json = JSON::XS->new->convert_blessed->encode( { ver => $^V } );
say $json'
Can also add a sub via string eval at runtime but that doesn't seem needed.
write the code reference to the class's symbol table at runtime
perl -wE'use JSON::XS; say $^V;
*{"version"."::"."TO_JSON"} = sub { return $_[0]->stringify };
$json = JSON::XS->new->convert_blessed->encode( { ver => $^V } );
say $json'
well, or really with strict in effect we need to allow the symbolic refs
perl -Mstrict -wE'use JSON::XS; say $^V;
NO_STRICT_REFS: {
no strict "refs";
my ($class, $method) = qw(version TO_JSON);
*{$class."::".$method} = sub { return $_[0]->stringify }
};
my $json = JSON::XS->new->convert_blessed->encode( { ver => $^V } );
say $json'
where I also added variables for class name and method, not necessary but better.
This is packaged for far nicer use in Sub::Install
use Sub::Install;
Sub::Install::install_sub({
code => sub { ... }, into => $package, as => $subname
});
There are expected defaults and a bit more in the module.
Or, of course by writing a wrapper class or some such but that's something yet else.
Blessed Perl objects can't be stored in JSON without extra steps (mentioned by the error).
print ref $^V; # version
A possible workaround:
my $j = Cpanel::JSON::XS->new->convert_blessed; # Allow stringification.
say 'Hash1: ', $j->encode(\%hash1);

Validate data from JSON having Boolean(true,false) not ("true","false"). And string ("12345") not (12345)

Need to validate the data form JSON file.
Example JSON:
Expected
{
id => "12345",
flag=> true
}
I need to throw error if the file having data like below
{
id => 12345,
flag=> "true"
}
I found this link for validating the string here
But if there any other solution, it would be helpful for me.
TIA
Is $val a String?
The linked answer won't help you differentiate "12345" from 12345. To check if a value is stored as a string, you can use
use B qw( svref_2object SVf_POK );
svref_2object(\$val)->FLAGS & SVf_POK
or die("Not a string\n");
This should work for the values from your JSON parser, but it won't work if you can expect dual vars, objects with overloads or magical scalars.
Is $val a Boolean?
The most popular JSON parsers use special objects to represent true and false in JSON.
With Cpanel::JSON::XS, you can use
Cpanel::JSON::XS::is_bool($val)
or die("Not a boolean\n");
JSON::XS and JSON::PP also provide some means of checking.

JSON decoding in PERL - Maintaining the original data type

I am writing a simple perl script to read JSON from a file and insert into MongoDB. But I am facing issues with json decoding.
All non-string values in my original json are getting converted to object type after decode_json.
Input JSON(only part of it since it's original is huge) -
{
"_id": 2006010100000801089,
"show_image" : false,
"event" : "publish",
"publish_date" :1136091600,
"data_version" : 1
}
JSON that gets inserted to MongoDB -
{
"_id": NumberLong("2006010100000801089"),
"show_image" : BinData(0,"MA=="),
"event" : "publish",
"publish_date" :NumberLong(1136091600),
"data_version" : NumberLong(1)
}
I am providing the custom _id for the documents, which I want to get converted to NumberLong type. That is working as expected as you can see from the JSON above. But notice how other non-string values for show_image, publish_date and data_version got converted to it's object representation.
Is there any way I can retain the original type for these values?
Perl code snipper that does the insert -
use MongoDB;
use MongoDB::OID;
use JSON;
use JSON::XS
while(my $record = <$source_file>) {
my $record_decoded = decode_json($record);
$db_collection->insert($record_decoded);
}
Perl version used v5.18.2.
I looked up JSON::XS docs but couldn't find a way to do this. Any help is appreciated. Thanks in advance!
I am very new to perl. Sorry if this is a trivial question.
I am providing the custom _id for the documents, which I want to get converted to NumberLong type. That is working as expected as you can see from the JSON above. But notice how other non-string values for show_image, publish_date and data_version got converted to it's object representation.
From your example all of the data types are actually matching aside from the boolean value for show_image which is currently being converted to binary data.
It is expected that numeric types are displayed as NumberLong or NumberInt when queried from the mongo shell. The mongo shell uses JavaScript, which only has a single numeric type of Number (64-bit floating point). Shell helpers like NumberLong() and NumberInt() are used to represent values in MongoDB's BSON data types that do not have a native JavaScript equivalent.
Referring to my sample JSON, I want value of show_image to be inserted as false instead of BinData(0,"MA==") and publish_date to be inserted as 1136091600 instead of NumberLong(1136091600)
While it's OK to insert publish_date as a Unixtime if that suits your use case, you may find it more useful to use MongoDB's Date type instead. There are convenience methods for querying dates including Date Aggregation Operators. FYI, date fields will be displayed in the mongo shell with an ISODate() wrapper.
The boolean value for show_image definitely needs an assist, though.
If you use Data::Dumper to inspect the result from decode_json(), you will see that the show_image field is a blessed object:
'show_image' => bless( do{\(my $o = 0)}, 'JSON::PP::Boolean' )
In order to get the expected boolean value in MongoDB, the recommended approach in the MongoDB module docs is to use the boolean module (see: MongoDB::DataTypes).
I couldn't find an obvious built-in option for JSON or JSON::XS to support serialising booleans to something other than the JSON emulated boolean class, but one solution would be to use the Data::Clean::Base module which is part of the Data::Clean::JSON distribution.
Sample snippet (excluding the MongoDB set up):
use Data::Clean::Base;
use boolean;
my $cleanser = Data::Clean::Base->new(
'JSON::XS::Boolean' => ['call_func', 'boolean::boolean'],
'JSON::PP::Boolean' => ['call_func', 'boolean::boolean']
);
while (my $record = <$source_file>) {
my $record_decoded = decode_json($record);
$cleanser->clean_in_place($record_decoded);
$db_collection->insert($record_decoded);
}
Sample record as saved in MongoDB 3.0.2:
{
"_id": NumberLong("2006010100000801089"),
"event": "publish",
"data_version": NumberLong("1"),
"show_image": false,
"publish_date": NumberLong("1136091600")
}
JSON data contains only (double-precision) numbers, strings, and the special values true, false, and null. They can be arranged in arrays or "objects" (hashes).
The MongoDB engine is converting these basic types into something more complex, but the original values are available in the hash referred to by $record_decoded, like so
$record_decoded->{_id}
$record_decoded->{show_image}
$record_decoded->{event}
$record_decoded->{publish_date}
$record_decoded->{data_version}
Is that what you wanted?
The object serialization documentation (particularly allow_tags) in JSON::XS may do something like what you want. Note, though, that this is not a standard JSON feature and will only work with JSON::XS.

Remove __CLASS__ From JSON Output of Moose Object In Perl

I'm working with moose objects in perl. I want to be able to covert the moose objects I make directly to JSON.
However, when I use use MooseX::Storage to covert the objects, it includes a hidden attribute that I don't know how to remove the "__CLASS__" .
Is there a way to remove this using MooseX::Storage ? (For now I am just using MooseX::Storage to covert it and using JSON to remove the "__ CLASS __ " attribute by going to a hash . ) The solution I am doing for now is a problem, because I have to do it everytime I get the JSON for every object(so when I write the JSON output to a file, to be loaded I have to make the changes everytime, and any referanced objects also have to be handled)
package Example::Component;
use Moose;
use MooseX::Storage;
with Storage('format' => 'JSON');
has 'description' => (is => 'rw', isa => 'Str');
1;
no Moose;
no MooseX::Storage;
use JSON;
my $componentObject = Example::Component->new;
$componentObject->description('Testing item with type');
my $jsonString = $componentObject->freeze();
print $jsonString."\n\n";
my $json_obj = new JSON;
my $perl_hash = $json_obj->decode ($jsonString);
delete ${$perl_hash}{'__CLASS__'};
$jsonString = $json_obj->encode($perl_hash);
print $jsonString."\n\n";
MooseX::Storage is not particularly suited to this task. It's designed to enable persistent storage of Moose objects (that's why it adds the __CLASS__ field) so they can be retrieved by your program later.
If your goal is to construct objects for a JSON API, then it would probably be much easier to just pass your object's hashref directly to JSON.pm.
use JSON -convert_blessed_universally;
my $json_obj = JSON->new->allow_blessed->convert_blessed;
my $jsonString = $json_obj->encode( $componentObject );
The -convert_blessed_universally option (in addition to being a mouthful) will cause JSON.pm to treat blessed references (objects) as ordinary Perl structures which can be translated to JSON directly.
EDIT: Looks like you have to add the allow_blessed and convert_blessed options to the JSON object also.

PowerShell changes return object's type

I am using PowerShell v3 and the Windows PowerShell ISE. I have the following function that works fine:
function Get-XmlNode([xml]$XmlDocument, [string]$NodePath, [string]$NamespaceURI = "", [string]$NodeSeparatorCharacter = '.')
{
# If a Namespace URI was not given, use the Xml document's default namespace.
if ([string]::IsNullOrEmpty($NamespaceURI)) { $NamespaceURI = $XmlDocument.DocumentElement.NamespaceURI }
# In order for SelectSingleNode() to actually work, we need to use the fully qualified node path along with an Xml Namespace Manager, so set them up.
[System.Xml.XmlNamespaceManager]$xmlNsManager = New-Object System.Xml.XmlNamespaceManager($XmlDocument.NameTable)
$xmlNsManager.AddNamespace("ns", $NamespaceURI)
[string]$fullyQualifiedNodePath = Get-FullyQualifiedXmlNodePath -NodePath $NodePath -NodeSeparatorCharacter $NodeSeparatorCharacter
# Try and get the node, then return it. Returns $null if the node was not found.
$node = $XmlDocument.SelectSingleNode($fullyQualifiedNodePath, $xmlNsManager)
return $node
}
Now, I will be creating a few similar functions, so I want to break the first 3 lines out into a new function so that I don't have to copy-paste them everywhere, so I have done this:
function Get-XmlNamespaceManager([xml]$XmlDocument, [string]$NamespaceURI = "")
{
# If a Namespace URI was not given, use the Xml document's default namespace.
if ([string]::IsNullOrEmpty($NamespaceURI)) { $NamespaceURI = $XmlDocument.DocumentElement.NamespaceURI }
# In order for SelectSingleNode() to actually work, we need to use the fully qualified node path along with an Xml Namespace Manager, so set them up.
[System.Xml.XmlNamespaceManager]$xmlNsManager = New-Object System.Xml.XmlNamespaceManager($XmlDocument.NameTable)
$xmlNsManager.AddNamespace("ns", $NamespaceURI)
return $xmlNsManager
}
function Get-XmlNode([xml]$XmlDocument, [string]$NodePath, [string]$NamespaceURI = "", [string]$NodeSeparatorCharacter = '.')
{
[System.Xml.XmlNamespaceManager]$xmlNsManager = Get-XmlNamespaceManager -XmlDocument $XmlDocument -NamespaceURI $NamespaceURI
[string]$fullyQualifiedNodePath = Get-FullyQualifiedXmlNodePath -NodePath $NodePath -NodeSeparatorCharacter $NodeSeparatorCharacter
# Try and get the node, then return it. Returns $null if the node was not found.
$node = $XmlDocument.SelectSingleNode($fullyQualifiedNodePath, $xmlNsManager)
return $node
}
The problem is that when "return $xmlNsManager" executes the following error is thrown:
Cannot convert the "System.Object[]" value of type "System.Object[]" to type "System.Xml.XmlNamespaceManager".
So even though I have explicitly cast my $xmlNsManager variables to be of type System.Xml.XmlNamespaceManager, when it gets returned from the Get-XmlNamespaceManager function PowerShell is converting it to an Object array.
If I don't explicitly cast the value returned from the Get-XmlNamespaceManager function to System.Xml.XmlNamespaceManager, then the following error is thrown from the .SelectSingleNode() function because the wrong data type is being passed into the function's 2nd parameter.
Cannot find an overload for "SelectSingleNode" and the argument count: "2".
So for some reason PowerShell is not maintaining the data type of the return variable. I would really like to get this working from a function so that I don't have to copy-paste those 3 lines all over the place. Any suggestions are appreciated. Thanks.
What's happening is PowerShell is converting your namespace manager object to a string array.
I think it has to do with PowerShell's nature of "unrolling" collections when sending objects down the pipeline. I think PowerShell will do this for any type implementing IEnumerable (has a GetEnumerator method).
As a work around you can use the comma trick to prevent this behavior and send the object as a whole collection.
function Get-XmlNamespaceManager([xml]$XmlDocument, [string]$NamespaceURI = "")
{
...
$xmlNsManager.AddNamespace("ns", $NamespaceURI)
return ,$xmlNsManager
}
More specifically, what is happening here is that your coding habit of strongly typing $fullyQualifiedModePath is trying to turn the result of the Get (which is a list of objects) into a string.
[string]$foo
will constrain the variable $foo to only be a string, no matter what came back. In this case, your type constraint is what is subtly screwing up the return and making it Object[]
Also, looking at your code, I would personally recommend you use Select-Xml (built into V2 and later), rather than do a lot of hand-coded XML unrolling. You can do namespace queries in Select-Xml with -Namespace #{x="..."}.