I'm reading a JSON in pyspark and seeing the below issue.
Column name "change(me)" contains invalid characters, please use alias to rename it
I have tried to use withColumnRenamed but that does not seem to help
df = spark.read.option("multiline","true").json("json_file")
df = df.withColumnRenamed("change(me)", "change_me")
Here is my sample json
{
"1": {
"task": [
"wakeup",
"getready"
]
},
"2": {
"task": [
"brush",
"shower"
]
},
"3": {
"task": [
"brush",
"shower"
]
},
"activites": ["standup", "play", "sitdown"],
"statuscheck": {
"time": 60,
"color": 1002,
"change(me)": 9898
},
"action": ["1", "2", "3", "4"]
}
when I check for columns in my dataframe, I do not see change(me) but it still complains of invalid character
Related
I have a table that contains a JSON column. Currently, the JSON column holds data which has the below schema :
Input or Source JSON looks like below :
"Q1": {
"id": 1,
"value": 12
},
"Q2": [{
"id": 1,
"value": 12
}, {
"id": 1,
"value": 12
}],
"Q3": {
"id": 1,
"additional": true
}
}
I have to perform a migration activity wherein I need to convert the Q1 key's value type from JSON object to JSON array. I don't see any utility functions which can help me in MySQL. Please suggest a suitable approach that can help me.
Result JSON should look like below
{
"Q1": [{
"id": 1,
"value": 12
}],
"Q2": [{
"id": 1,
"value": 12
}, {
"id": 1,
"value": 12
}],
"Q3": {
"id": 1,
"additional": true
}
}
I am working with a specific API that returns a JSON that looks like the below sample.
I want to get both values that contain the #text and #attr but I get error messages in typescript when I try to get the values.
try using,
album[0]["#attr"]
album[0]["artist"]["#text"]
Hey for JSON you can use get details by its attribute name in it and it's the same for all-weather it starts with # or # it will be the same.
See below code to get the value of your specified key:
Sample JSON:
{
"weeklyalbumchart": {
"album": [
{
"artist": {
"mbid": "data",
"#text": "Flying Lotus"
},
"mbid": "data",
"url": "",
"name": "",
"#attr": {
"rank": "1"
},
"playcount": "21"
},
{
"artist": {
"mbid": "data",
"#text": "Flying Lotus"
},
"mbid": "data",
"url": "",
"name": "",
"#attr": {
"rank": "1"
},
"playcount": "21"
}
]
}
}
Read JSON:
#attr ===> json["weeklyalbumchart"]["album"][0]["#attr"]
#text ===> json["weeklyalbumchart"]["album"][0]["artist"]["#text"]
Hope this will help you to understand it.
Beginner RegExp question. I have a JSON in my NiFi ExtractText and there are 2 fields I want to extract. How would I use a regex to do this?
[
{
"id": "12erf3-312331-233"
},
[
{
"id": "1234",
"id2": "1234",
"id3": "1234"
},
{
"id": "1234",
"id2": "1234",
"id3": "1234"
},
{
"id": "1234",
"id2": "1234",
"id3": "1234"
},
{
"id": "1234",
"id2": "1234",
"id3": "5555"
}
]
]
get the first id: "12erf3-312331-233" (this is a uuid)
get all the second array: [ { "id": "1234" ... "id3": "5555" } ]
To achieve what you need you may use ReplaceText processor. Have a look to the configuration below. Search Value would be (?s).*("id": )(".*").*}.*(\[.*\]).*\] and the Replacement Value: {$1$2,data:$3}. Pay attention to pick Evaluation Mode as "Entire text".
To easily check regex you can use link i have used: https://regex101.com/r/p6T2Vw/1
When I try to save my changes from the Google Apps Script file I get the following error:
Missing ] after element list.
This is my code:
var request = {
"name": "Name",
"id": 3,
"rules":[
{
"name": "Nested",
"tags": [
{
"tagId": 1,
"variables":[
[
"variable": "Var1"
]
],
"condition": false,
},
{
"tagId": 1,
"condition": false,
}
],
"ruleSetId": 3,
}
]
}
The error indicates that the problem is on the line that contains "variable": allScopes[i].variable, but I can't find where the problem is...
This is an example with the JSON object that I need to build:
Note that you are trying to use the array literal to construct an object:
"variables":[
[
"variable": "Var1"
]
],
As it looks like you need an object, not an array here, replace the inner [] with {}:
"variables":[
{
"id": null,
...,
"value": ".*"
}
],
I'm trying to save this to a Cloudant database:
[
{
"_id": "document_one",
"heynow": [
{
"Name": "one",
"Duration": 2,
"DurationUnit": "Hours"
},
{
"Name": "two",
"Duration": 40,
"DurationUnit": "Minutes"
}
]
},
{
"_id": "document_two",
"heynow": [
{
"Name": "three",
"Duration": 2,
"DurationUnit": "Hours"
},
{
"Name": "four",
"Duration": 40,
"DurationUnit": "Minutes"
}
]
}
]
But apparently it doesn't like the outer brackets because it's telling me:
"error":"bad_request","reason":"Document must be a JSON object"
JSONLint says it's valid, so I guess I'm asking if anyone knows how to format this so it can be entered into a Couchdb because the outer brackets seem to be causing problems.
CouchDB has a bulk documents API;
https://wiki.apache.org/couchdb/HTTP_Bulk_Document_API