I have the following terraform config:
resource "aws_dynamodb_table_item" "my_table" {
table_name = aws_dynamodb_table.my_table.name
hash_key = aws_dynamodb_table.my_table.hash_key
item = <<ITEM
{
"id": {"S": "nameAndCodes"},
"data": {"S": "[
{
"code": "03",
"displayName": "name1"
},
{
"code": "04",
"displayName": "name2"
}
]"}
}
ITEM
}
When the plan stage executes I receive the error:
Error: Invalid format of "item": Decoding failed: invalid character '\r' in string literal
The only way i can get this to work is to make the whole json a single line as follows:
"data": {"S": "[{\"code\": \"03\", \"displayName\": \"name1\"},{\"code\": \"04\", \"displayName\": \"name2\"}]"
This looks very ugly and difficult to manage.
Does anyone know how I can enter a multiline JSON inside a <<ITEM block?
To resolve that issue, You can use the jsonencode function to set the item value and put entire JSON object in there. Here is an example in Terraform from my project which creates a DynamoDB table and put an initial item.
resource "aws_dynamodb_table" "customer_table" {
name = "customer"
billing_mode = "PAY_PER_REQUEST"
hash_key = "customerId"
stream_enabled = false
attribute {
name = "customerId"
type = "S"
}
}
resource "aws_dynamodb_table_item" "customer_table_item" {
table_name = aws_dynamodb_table.customer_table.name
hash_key = aws_dynamodb_table.customer_table.hash_key
depends_on = [aws_dynamodb_table.customer_table]
item = jsonencode({
"customerId" : {
"S" : "1"
},
"firstName" : {
"S" : "John"
},
"lastName" : {
"S" : "Doe"
},
})
}
commands:
terrform init
terraform fmt
terraform plan
terraform apply
Related
I'm using pydantic 1.6.1 and fastapi 0.61.1 with Python 3.8.3. Here's how I've tried defining the models:
class UserBase(BaseModel):
name: str
class UserCreate(UserBase):
password: str
class UserInfo(UserBase):
id: str
group: Optional[GroupInfo] = None
The issue I'm having with this setup is that the schema is built such that the name attribute is on top of the remaining attributes like so - in this case, while using UserInfo as the endpoint's response_model:
[
{
"name": "string",
"id": "string",
"group": {
"name": "string"
}
}
]
Yet I'd like to be able to set them up like this:
[
{
"id": "string",
"name": "string",
"group": {
"name": "string"
}
}
]
Is there a way I can manually set a custom order for the attributes in the JSON response's schema?
As I mentioned in my comment, it doesn't really matter the order of the JSON, but when it comes to schema generation, it can be helpful.
You need to decouple the id field from UserInfo model as
class UserID(BaseModel):
id: str
class UserInfo(UserBase, UserID): # `UserID` should be second
group: Optional[GroupInfo] = None
This will generate the following JSON schema,
{
"id": "string",
"name": "string",
"group": {
"name": "string"
}
}
Bit more ugly solution (IMO)
patch the .__fields__ as
class UserBase(BaseModel):
name: str
class UserCreate(UserBase):
password: str
class UserInfo(UserBase):
id: str
group: Optional[GroupInfo] = None
fields = UserInfo.__fields__.copy()
UserInfo.__fields__ = {"id": fields.pop("id"), **fields}
I'm getting a JSON object over the network, as a String. I'm then using Circe to parse it. I want to add a handful of fields to it, and then pass it on downstream.
Almost all of that works.
The problem is that my "adding" is really "overwriting". That's actually ok, as long as I add an empty object first. How can I add such an empty object?
So looking at the code below, I am overwriting "sometimes_empty:{}" and it works. But because sometimes_empty is not always empty, it results in some data loss. I'd like to add a field like: "custom:{}" and then ovewrite the value of custom with my existing code.
Two StackOverflow posts were helpful. One worked, but wasn't quite what I was looking for. The other I couldn't get to work.
1: Modifying a JSON array in Scala with circe
2: Adding field to a json using Circe
val js: String = """
{
"id": "19",
"type": "Party",
"field": {
"id": 1482,
"name": "Anne Party",
"url": "https"
},
"sometimes_empty": {
},
"bool": true,
"timestamp": "2018-12-18T11:39:18Z"
}
"""
val newJson = parse(js).toOption
.flatMap { doc =>
doc.hcursor
.downField("sometimes_empty")
.withFocus(_ =>
Json.fromFields(
Seq(
("myUrl", Json.fromString(myUrl)),
("valueZ", Json.fromString(valueZ)),
("valueQ", Json.fromString(valueQ)),
("balloons", Json.fromString(balloons))
)
)
)
.top
}
newJson match {
case Some(v) => return v.toString
case None => println("Failure!")
}
We need to do a couple of things. First, we need to zoom in on the specific property we want to update, if it doesn't exist, we'll create a new empty one. Then, we turn the zoomed in property in the form of a Json into JsonObject in order to be able to modify it using the +: method. Once we've done that, we need to take the updated property and re-introduce it in the original parsed JSON to get the complete result:
import io.circe.{Json, JsonObject, parser}
import io.circe.syntax._
object JsonTest {
def main(args: Array[String]): Unit = {
val js: String =
"""
|{
| "id": "19",
| "type": "Party",
| "field": {
| "id": 1482,
| "name": "Anne Party",
| "url": "https"
| },
| "bool": true,
| "timestamp": "2018-12-18T11:39:18Z"
|}
""".stripMargin
val maybeAppendedJson =
for {
json <- parser.parse(js).toOption
sometimesEmpty <- json.hcursor
.downField("sometimes_empty")
.focus
.orElse(Option(Json.fromJsonObject(JsonObject.empty)))
jsonObject <- json.asObject
emptyFieldJson <- sometimesEmpty.asObject
appendedField = emptyFieldJson.+:("added", Json.fromBoolean(true))
res = jsonObject.+:("sometimes_empty", appendedField.asJson)
} yield res
maybeAppendedJson.foreach(obj => println(obj.asJson.spaces2))
}
}
Yields:
{
"id" : "19",
"type" : "Party",
"field" : {
"id" : 1482,
"name" : "Anne Party",
"url" : "https"
},
"sometimes_empty" : {
"added" : true,
"someProperty" : true
},
"bool" : true,
"timestamp" : "2018-12-18T11:39:18Z"
}
I'd like to select all elements with a certain match in the name of the property.
For example, all the properties whose name starts with 'pass' from this json:
{
"firstName": "John",
"lastName" : "doe",
"age" : 50,
"password" : "1234",
"phoneNumbers": [
{
"type" : "iPhone",
"number": "0123-4567-8888",
"password": "abcd"
},
{
"type" : "home",
"number": "0123-4567-8910",
"password": "fghi"
}
]
}
Would result something like this:
[
"1234",
"abcd",
"fghi"
]
I don't want filter by values, only by property names. Is it possible using jsonpath?
I'm using the method SelectTokens(string path) of Newtonsoft.Json.Linq
No, JSONPath defines expressions to traverse through a JSON document to reach to a subset of the JSON. It cannot be used when you don't know the exact property names.
In your case you need property values whose name starts with a specific keyword. For that, you need to traverse the whole JSON text and look for the property names which start with pass having a string type
var passwordList = new List<string>();
using (var reader = new JsonTextReader(new StringReader(jsonText)))
{
while (reader.Read())
{
if(reader.TokenType.ToString().Equals("PropertyName")
&& reader.ValueType.ToString().Equals("System.String")
&& reader.Value.ToString().StartsWith("pass"))
{
reader.Read();
passwordList.Add(reader.Value.ToString());
}
}
passwordList.ForEach(i => Console.Write("{0}\n", i));
}
I am trying to loop through Jsonnode but root jsonNode is duplicating the data.
Trying to figure out but not sure where i am missing the issue. Will try to explain the issue below.
I have to Jackson API.
Json block is:
{
"queries": [
{
"id": "keyword",
"values": [
"test"
]
},{
"id": "include",
"values": [
false
]
}
]
}
My block of Java code is Iterator fieldNames = root.fieldNames();
while (fieldNames.hasNext()) {
String fieldName = fieldNames.next();
if (fieldName.equalsIgnoreCase("queries")) {
nameNode =root.get(fieldName);
}
JsonNode nameNode = root.get("queries");
for (JsonNode node : nameNode) {
JsonNode elementId = node.path("id").asText();
if (!elementId.isEmpty() && elementId.equalsIgnoreCase("include")) {
check = true;
include = node;
}
}
When debug comes to line for (JsonNode node : nameNode) { , node value is "id": "keyword", "values": [ "test" ] and nameNode is the json shown above but when it comes to next line which is " node.path("id").asText();"
nameNode variable appends "id": "keyword","values": [ "test" ] 2 times.
Now the json is the original json with "id": "keyword","values": [ "test" ] appended 2 times and gives concurrentModificationException.
change your variable node to objNode because node may be predifined value in jackson and you can also try to make for each variable to final
I have a json output like below,
{
"took":3,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},
"hits":{"total":2,"max_score":1.0,
"hits":
[
{
"_index":"management",
"_type":"",
"_id":"/home/myfld1/myid1",
"_score":1.0,
"_source" :
{
"newslides": "User Mgmt1 ",
"metaData":
{
"fileName": "file1",
}
}
},
{
"_index":"management",
"_type":"",
"_id":"/home/myfld3/myid3",
"_score":1.0,
"_source" :
{
"newslides": "User mgmt2 ",
"metaData":
{
"fileName": "file2",
}
}
}
]
}
}
I am trying to get the "fileName" field from the above json. I have tried like this,
for filenames in response["hits"]["hits"][0]["_source"]["newslides"]["metaData"]:
filearray.append(filenames["fileName"])
But i am getting one error like below,
for filenames in response["hits"]["hits"][0]["_source"]["newslides"]["metaData"]:
TypeError: string indices must be integers
Please help me to get that file name.
It is simply a dictionary with some lists and other dictionaries in it. Just be careful and go through the elements iteratively:
filearray = []
for x in response["hits"]["hits"]:
filearray.append(x["_source"]["metaData"]["fileName"])
or in one line:
filearray = [x["_source"]["metaData"]["fileName"] for x in response["hits"]["hits"]]