Encode complex JSON structure using Perl - json

Question

I want to encode the result of a MySQL query into a JSON string using JSON::XS. The JSON string needs to look like this
{
"database" : "dbname"
"retentionPolicy" : "mytest",
"tags" : {
"type" : "generate",
"location" : "total",
"source" : "ehz"
},
"points" : [{
"precision" : "ms",
"timestamp" : "ts1",
"name" : "power",
"values" : {
"value" : "val1"
}
}, {
"precision" : "ms",
"timestamp" : "ts2",
"name" : "power",
"values" : {
"value" : "val2"
}
}, {
"precision" : "ms",
"timestamp" : "ts3",
"name" : "power",
"values" : {
"value" : "val3"
}
}
]
}
The points array with each point's values element is giving me immense headaches.
Here is the code block that generates the JSON
my %json_body = (
'database' => $db_name,
'retentionPolicy' => $retention,
'tags' => {
'source' => $metric_source,
'type' => $metric_type,
'location' => $metric_location
}
);
# loop through mysql result
while ( ($timestamp, $value) = $query->fetchrow_array() ) {
my %json_point1 = (
'name' => $series_name,
'timestamp' => ($timestamp * 1),
'precision' => "ms"
);
%json_point2 = ('value' => $value);
%json_values = (%json_point1, 'values' => \%json_point2);
push(#all_values, \%json_values);
}
$query->finish();
# Encode json
my %json_data = (%json_body, "points" => \#all_values);
$influx_json = encode_json(\%json_data);
I think the line push(#all_values, \%json_values) is my problem. If I pass %json_data as a hash reference, only the last value from the while loop is retained. If I use %json_values directly, the encoded JSON is messed up because it loses the structure.
Any hint would be appreciated. And please bear with me: this array and hash references are already making my head explode.

Answer 1

I'm pretty sure you problem will be because you're using a globally scoped hash for %json_point and %json_point2.
You see, the root of this is - you simply don't get a list of hashes. You get a list of hash references.
So the problem here is - when you push a reference to your hash into #all_values - you're pushing the same reference each time. But then you're overwriting the contents of the hash that you're referencing.
Try this:
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
my %hash_thing;
my #all_values;
for ( 1..3 ) {
%hash_thing = ( "test" => $_ );
push ( #all_values, \%hash_thing ) ;
}
print join ( "\n", #all_values );
print Dumper \#all_values;
And you'll see you have the same 'value' 3 times:
HASH(0x74478c)
HASH(0x74478c)
HASH(0x74478c)
And so if you dump it, then of course - you don't get the right array - and so your encoded JSON doesn't work either.
$VAR1 = [
{
'test' => 3
},
$VAR1->[0],
$VAR1->[0]
];
The simplest fix is to use my to scope the hashes to the loop. (And turn on use strict; and use warnings if you haven't.)
Alternatively, you can use a hash reference like this:
my #all_values;
my $hash_ref;
for ( 1..3 ) {
$hash_ref = { "test" => $_ };
push ( #all_values, $hash_ref ) ;
}
print #all_values;
print Dumper \#all_values;
Because $hash_ref is a scalar, and it's a reference to an anonymous hash, it can be inserted into the array by value, rather than by reference.