handling a well-formed JSON file of an array of objects - json

A JSON string string passes the jsonlint test.
response = [
{
"article" : {
"info" : {
"initial" : {
"articleIds" : [
"7461221587662919569"
],
}
},
"text" : "where they would 'transfer to' next.",
"lang" : "en",
}
},
{
"article" : {
"info" : {
"initial" : {
"articleIds" : [
"6613144915874808065"
],
}
},
"text" : "produto regional.",
"lang" : "pt"
}
}
]
However, after processing
require 'json'
file = File.read('/Users/main/jugg//article_samples.js')
data_hash = JSON.parse(file)
One is left with an array, whereas more frequently a hash with a name labels a subsequent array, where one works with that nomenclature such as response['data']
But in this case the array is not accessible via response[0]. How can this be considered as an array in order to process each individual element collection.each do |member|?
A curiosity: data_hash.class => NilClass

The response = ... code from article_samples.js is JavaScript, not JSON. This initializes a variable named response with a JavaScript array.
To use this as JSON, then rename the file to article_samples.json and remove response = from the file. The first line should start with [.
Now your second block of code should work just fine as long as the article_samples.json file is in the correct path.
On a side note, I suggest that you find a way to make the path more flexible. The way you have it currently hard coded is tied directly to your current machine's file system. This won't work if you want to run this code from another machine because the folder /Users/main/jugg probalby won't exist.
If this is a web server with ruby on rails, then one solution is to create an environment variable with the path where this file is stored.

Related

Workaround to add JSON with errors to mongodb atlas collection

In my database class we were given an assignment to work with two JSON files (add them to a mongodb atlas collection and query certain results)
Both JSON files had "errors" the first being :
{ "_id" : { "$oid" : "50b59cd75bed76f46522c34e" }, "student_id" : 0, "class_id" : 2, "scores" : [ { "type" : "exam", "score" : 57.92947112575566 }, { "type" : "quiz", "score" : 21.24542588206755 }, { "type" : "homework", "score" : 68.19567810587429 }, { "type" : "homework", "score" : 67.95019716560351 }, { "type" : "homework", "score" : 18.81037253352722 } ] }
and the second being :
{"_id":0,"name":"aimee Zank","scores":[{"score":1.463179736705023,"type":"exam"},{"score":11.78273309957772,"type":"quiz"},{"score":35.8740349954354,"type":"homework"}]},
{"_id":1,"name":"Aurelia Menendez","scores":[{"score":60.06045071030959,"type":"exam"},{"score":52.79790691903873,"type":"quiz"},{"score":71.76133439165544,"type":"homework"}]},
I fixed error 1 by removing the $oid and replacing it with just oid: as there was an error trying to add objects with $oid as a value to my collection. I also needed to add everything to an array.
I fixed the second by putting the entire object inside an array [].
When I asked my professor why these errors were in the JSON files and if it was on purpose, he said that they were there on for a reason and that we needed to find a "work around".
I am curious what work around there is to load JSON data that is incorrect into a collection? I am at a complete loss as to what he expected. Is there some way I can just load individual objects line by line from the JSON file to the collection?
This is how I loaded the JSON data after fixing the files directly:
const fs = require('fs');
var data = JSON.parse(fs.readFileSync("./students.json"));
JSON.stringify(data);
const database = "college";
const collection = "students";
use(database);
db.students.drop();
db.createCollection(collection);
db.students.insertMany(data);
--- All the importing of data should be done in VS Code and not using --mongodb import
And a side note that this assignment has since passed so I am not asking for help in completing my homework, simply trying to see if there was something I could of done that would not of required me to edit the JSON file itself. My professor has not responded to me regarding this question.

Create Terraform resources out of JSON values

I am looking for a way to generate Terraform code based on JSON values.
Imagine I have a JSON file with the following structure:
{
"settings": [
{
"conf": [
{
"setting": "DeploymentPolicy",
"namespace": "aws:elasticbeanstalk:command",
"value": "AllAtOnce"
},
{
"setting": "BatchSize",
"namespace": "aws:elasticbeanstalk:command",
"value": "30"
},
{
"setting": "BatchSizeType",
"namespace": "aws:elasticbeanstalk:command",
"value": "Percentage"
}
]
}
]
}
What I want to do is the following:
Creating a working Terraform resource based on the JSON file values, e.g. a beanstalk environment like this:
resource "aws_elastic_beanstalk_environment" "app_prod" {
name = "${aws_elastic_beanstalk_application_version.app.name}-prod"
application = aws_elastic_beanstalk_application.app.name
solution_stack_name = data.aws_elastic_beanstalk_solution_stack.latest_linux_java.name
wait_for_ready_timeout = "10m"
version_label = aws_elastic_beanstalk_application_version.app.name
# Elastic beanstalk configuration
setting {
name = "DeploymentPolicy"
namespace = "aws:elasticbeanstalk:command"
value = "AllAtOnce"
}
setting {
name = "BatchSize"
namespace = "aws:elasticbeanstalk:command"
value = "30"
}
...
}
Therefore I have to create the settings block in HCL (Terraform configuration) based on the JSON values.
This means the JSON file above should result in:
setting {
name = "DeploymentPolicy"
namespace = "aws:elasticbeanstalk:command"
value = "AllAtOnce"
}
setting {
name = "BatchSize"
namespace = "aws:elasticbeanstalk:command"
value = "30"
}
setting {
name = "BatchSizeType"
namespace = "aws:elasticbeanstalk:command"
value = "Percentage"
}
As you can see, the structure of JSON and HCL is very similar, but not identical. See e.g. settings, conf, or setting instead of name in the JSON.
A possible approach would be to read the JSON values and store them in an array or a map. But I have no idea how I could generate valid HCL and inject it in the desired part of the resource. Furthermore I tried to use a template but Terraform does not support the looping functionality that I need to iterate over the settings.
To sum up:
Input is a JSON file that must be read
JSON contains settings (besides other information)
The number of settings can differ
Somehow I have to generate a settings block
Somehow I have to inject this settings blok in the resource
Does anyone have an idea how to do that? Any other approaches?
Thanks a lot!
Assuming that your JSON object were in a file called settings.json inside your module directory, you could do something like this:
locals {
environment_settings = jsondecode(file("${path.module}/settings.json")).settings[0].conf[0]
}
resource "aws_elastic_beanstalk_environment" "app_prod" {
name = "${aws_elastic_beanstalk_application_version.app.name}-prod"
application = aws_elastic_beanstalk_application.app.name
solution_stack_name = data.aws_elastic_beanstalk_solution_stack.latest_linux_java.name
wait_for_ready_timeout = "10m"
version_label = aws_elastic_beanstalk_application_version.app.name
dynamic "setting" {
for_each = local.environment_settings
content {
namespace = setting.value.namespace
name = setting.value.setting
value = setting.value.value
}
}
}
This special dynamic block is a sort of macro to create repeated setting blocks, each one correlating with one element of the collection given in for_each.
You can do whatever transformations of the input you need using Terraform's expression language in the locals block to ensure that the local.environment_settings value contains one element for each setting block you will generate, and then in the content nested block tell Terraform how to populate the setting arguments based on those element values.

Using Laravel, is there a way to run validation on one ajax call with data for multiple models?

Assuming one were to post multiple data sets of one model at the time through JSON, it is possible to insert these using Eloquent's Model::create() function. However in my case I'll also need to validate this data.
The Validator only takes a Request object as input, and as far as I've seen I can't create a new Request instance with only one model.
Assuming this would be the input data (JSON), and index is the value for the browser to know what data belongs to an what item (as they have no unique ID assigned at the point of creation)
[
{
"index" : 1,
"name" : "Item 1",
"value" : "Some description"
},
{
"index" : 2,
"name" : "Item 2",
"value" : "Something to describe item 2"
},
(and so on)
]
Every object in the root array needs to be ran through the same validator. The rules of it are defined in Model::$rules (public static array).
Would there be a way to run the validator against every item, and possibly capture the errors per item?
You can utilize Validator for manual validation:
...
use Validator;
...
$validator = Validator::make(
json_decode($data, true), // where $data contains your JSON data string
[
// List your rules here using wildcard syntax.
'*.index' => 'required|integer',
'*.name' => 'required|min:2',
...
],
[
// Array of messages for validation errors.
...
],
[
// Array of attribute titles for validation errors.
...
]
);
if ($validator->fails()) {
// Validation failed.
// $validator->errors() will return MessageBag with what went wrong.
...
}
You can read more about validating arrays here.

tMongoDBOutput - Configuring JSON tree

Trying to configure JSON tree for tMongoDBOutput. Only 1 element is getting created in subelement array. Can someone please give a example of configuring the JSON tree. Requirement is one document can have multiple nested sub documents. Medical_records can have multiple sub documents, but only 1 sub document is getting created now skipping the rest.
Resulting JSON in MongoDB as follows
{
"first_name" : "testname",
"middle_name" : [],
"last_name" : "test",
"medical_records" : [
{
"dateofuploading" : "2016-09-29 12:49:21.5",
"filename" : "demo.pdf",
"isautogenerated" : "1",
"recordid" : "123"
}
]
}
enter image description hereIf you want to have multiple sub documents in array,You need to use the group by operation in the advance settings for the element inside the array .

Push new items into JSON array

Let's say this is the table inside my collection:
{
"_id" : ObjectId("557cf6bbd8efe38c627bffdf"),
"name" : "John Doe",
"rating" : 9,
"newF" : [
"milk",
"Eggs",
"Beans",
"Cream"
]
}
Once a user types in some input, it is sent to my node server, and my node server then adds that item to the list "newF", which is then sent back to my MongoDB and saved.
I'm trying to use update, which can successfully change the values inside of this table, but I'm not sure how to add new items onto that list. I did it with $push inside the MongoDB shell, but not sure how to do it on node.
Here's a snippet of my code:
db.collection('connlist').update({ _id: new ObjectId("e57cf6bb28efe38c6a7bf6df")}, { name: "JohnDoe", rating: 9, newF: ["Milk, Eggs", "Beans"] }, function(err,doc){
console.log(doc);
});
Well the syntax for adding new items is just the same as in the shell:
// make sure you actually imported "ObjectID"
var ObjectId = require('mongodb').ObjectID;
db.collection('conlist').update(
{ "_id": new ObjectId("e57cf6bb28efe38c6a7bf6df") },
{ "$push": { "newF": { "$each": [ "cream", "butter" ] } } },
function(err,numAffected) {
// do something in the callback
}
)
Or perhaps use .findOneAndUpdate() if you want to return the modified document instead of just making the alteration.
Of course use $push and possibly with $each which allows multiple array elements to be added when adding to an array. If you want "unique" items then use $addToSet where your operation allows.
And generally speaking for other items you should use $set or other operators in the update portion of your document. Without these operators you are just "replacing" the document content with whatever structure you place in the "update" portion of your statement.