I've create a JSON file, and I need to be able to share the file via email with other collaborators. However, although there are plenty of topics available on handling JSON objects in the R workspace, there are virtually no resources discussing how to actually export a JSON object to a .JSON file.
Here's a simple example:
list1 <- vector(mode="list", length=2)
list1[[1]] <- c("a", "b", "c")
list1[[2]] <- c(1, 2, 3)
exportJson <- toJSON(list1)
## Save the JSON to file
save(exportJson, file="export.JSON")
## Attempt to read in the JSON
library("rjson")
json_data <- fromJSON(file="export.JSON")
The final line, attempting to read in the JSON file, results in an error: "Error in fromJSON(file = "export.JSON") : unexpected character 'R'"
Obviously the save() function is not the way to go, but after extensive googling, I have found nothing that says how to export the JSON to a file. Any help would be greatly appreciated.
You can use write:
library(RJSONIO)
list1 <- vector(mode="list", length=2)
list1[[1]] <- c("a", "b", "c")
list1[[2]] <- c(1, 2, 3)
exportJson <- toJSON(list1)
> exportJson
[1] "[\n [ \"a\", \"b\", \"c\" ],\n[ 1, 2, 3 ] \n]"
write(exportJson, "test.json")
library("rjson")
json_data <- fromJSON(file="test.json")
> json_data
[[1]]
[1] "a" "b" "c"
[[2]]
[1] 1 2 3
There is also the jsonlite package:
library(jsonlite)
exportJSON <- toJSON(list1)
write(exportJSON, "test.json")
list2 <- fromJSON("test.json")
identical(list1, list2)
Related
I have a list of lists which are of variable length. The first value of each nested list is the key, and the rest of the values in the list will be the array entry. It looks something like this:
[[1]]
[1] "Bob" "Apple"
[[2]]
[1] "Cindy" "Apple" "Banana" "Orange" "Pear" "Raspberry"
[[3]]
[1] "Mary" "Orange" "Strawberry"
[[4]]
[1] "George" "Banana"
I've extracted the keys and entries as follows:
keys <- lapply(x, '[', 1)
entries <- lapply(x, '[', -1)
but now that I have these, I don't know how I can associate a key:value pair in R without creating a matrix first, but this is silly since my data don't fit in a rectangle anyway (every example I've seen uses the column names from a matrix as the key values).
This is my crappy method using a matrix, assigning rownames, and then using jsonLite to export to JSON.
#Create a matrix from entries, without recycling
#I found this function on StackOverflow which seems to work...
cbind.fill <- function(...){
nm <- list(...)
nm <- lapply(nm, as.matrix)
n <- max(sapply(nm, nrow))
do.call(cbind, lapply(nm, function (x)
rbind(x, matrix(, n-nrow(x), ncol(x)))))
}
#Call said function
matrix <- cbind.fill(entries)
#Transpose the thing
matrix <- t(matrix)
#Set column names
colnames(matrix) <- keys
#Export to json
json<-toJSON(matrix)
The result is good, but the implementation sucks. Result:
[{"Bob":["Apple"],"Cindy":["Apple","Banana","Orange","Pear","Raspberry"],"Mary":["Orange","Strawberry"],"George":["Banana"]}]
Please let me know of better ways that might exist to accomplish this.
How about:
names(entries) <- unlist(keys)
toJSON(entries)
Consider the following lapply() approach:
library(jsonlite)
entries <- list(c('Bob', 'Apple'),
c('Cindy', 'Apple', 'Banana', 'Orange','Pear','Raspberry'),
c('Mary', 'Orange', 'Strawberry'),
c('George', 'Banana'))
# ITERATE ALL CONTENTS EXCEPT FIRST
inner <- list()
nestlist <- lapply(entries,
function(i) {
inner <- i[2:length(i)]
return(inner)
})
# NAME EACH ELEMENT WITH FIRST ELEMENT
names(nestlist) <- lapply(entries, function(i) i[1])
#$Bob
#[1] "Apple"
#$Cindy
#[1] "Apple" "Banana" "Orange" "Pear" "Raspberry"
#$Mary
#[1] "Orange" "Strawberry"
#$George
#[1] "Banana"
x <- toJSON(list(nestlist), pretty=TRUE)
x
#[
# {
# "Bob": ["Apple"],
# "Cindy": ["Apple", "Banana", "Orange", "Pear", "Raspberry"],
# "Mary": ["Orange", "Strawberry"],
# "George": ["Banana"]
# }
#]
I think this has already been sufficiently answered but here is a method using purrr and jsonlite.
library(purrr)
library(jsonlite)
sample_data <- list(
list("Bob","Apple"),
list("Cindy","Apple","Banana","Orange","Pear","Raspberry"),
list("Mary","Orange","Strawberry"),
list("George","Banana")
)
sample_data %>%
map(~set_names(list(.x[-1]),.x[1])) %>%
toJSON(auto_unbox=TRUE, pretty=TRUE)
I have some JSON that looks like this:
"total_rows":141,"offset":0,"rows":[
{"id":"1","key":"a","value":{"SP$Sale_Price":"240000","CONTRACTDATE$Contract_Date":"2006-10-26T05:00:00"}},
{"id":"2","key":"b","value":{"SP$Sale_Price":"2000000","CONTRACTDATE$Contract_Date":"2006-08-22T05:00:00"}},
{"id":"3","key":"c","value":{"SP$Sale_Price":"780000","CONTRACTDATE$Contract_Date":"2007-01-18T06:00:00"}},
...
In R, what would be the easiest way to produce a scatter-plot of SP$Sale_Price versus CONTRACTDATE$Contract_Date?
I got this far:
install.packages("rjson")
library("rjson")
json_file <- "http://localhost:5984/testdb/_design/sold/_view/sold?limit=100"
json_data <- fromJSON(file=json_file)
install.packages("plyr")
library(plyr)
asFrame <- do.call("rbind.fill", lapply(json_data, as.data.frame))
but now I'm stuck...
> plot(CONTRACTDATE$Contract_Date, SP$Sale_Price)
Error in plot(CONTRACTDATE$Contract_Date, SP$Sale_Price) :
object 'CONTRACTDATE' not found
How to make this work?
Suppose you have the following JSON-file:
txt <- '{"total_rows":141,"offset":0,"rows":[
{"id":"1","key":"a","value":{"SP$Sale_Price":"240000","CONTRACTDATE$Contract_Date":"2006-10-26T05:00:00"}},
{"id":"2","key":"b","value":{"SP$Sale_Price":"2000000","CONTRACTDATE$Contract_Date":"2006-08-22T05:00:00"}},
{"id":"3","key":"c","value":{"SP$Sale_Price":"780000","CONTRACTDATE$Contract_Date":"2007-01-18T06:00:00"}}]}'
Then you can read it as follows with the jsonlite package:
library(jsonlite)
json_data <- fromJSON(txt, flatten = TRUE)
# get the needed dataframe
dat <- json_data$rows
# set convenient names for the columns
# this step is optional, it just gives you nicer columnnames
names(dat) <- c("id","key","sale_price","contract_date")
# convert the 'contract_date' column to a datetime format
dat$contract_date <- strptime(dat$contract_date, format="%Y-%m-%dT%H:%M:%S", tz="GMT")
Now you can plot:
plot(dat$contract_date, dat$sale_price)
Which gives:
If you choose not to flatten the JSON, you can do:
json_data <- fromJSON(txt)
dat <- json_data$rows$value
sp <- strtoi(dat$`SP$Sale_Price`)
cd <- strptime(dat$`CONTRACTDATE$Contract_Date`, format="%Y-%m-%dT%H:%M:%S", tz="GMT")
plot(cd,sp)
Which gives the same plot:
I found a way that doesn't discard the field names:
install.packages("jsonlite")
install.packages("curl")
json <- fromJSON(json_file)
r <- json$rows
At this point r looks like this:
> class(r)
[1] "data.frame"
> colnames(r)
[1] "id" "key" "value"
After some more Googling and trial-and-error I landed on this:
f <- r$value
sp <- strtoi(f[["SP$Sale_Price"]])
cd <- strptime(f[["CONTRACTDATE$Contract_Date"]], format="%Y-%m-%dT%H:%M:%S", tz="GMT")
plot(cd,sp)
And the result on my full data-set...
i have three JSON files
json1 contains [[1,5],[5,7],[8,10]]
json2 contains [[5,6],[4,5],[5,8]]
json3 contains [[4,7],[3,4],[4,8]]
I want to merge them into one single file jsonmerge:
[[[1,5],[5,7],[8,10]],[[5,6],[4,5],[5,8]],[[4,7],[3,4],[4,8]]]
I tried concatenate but it gave results in this format
[[5,6],[4,5],[5,8]],
[[5,6],[4,5],[5,8]],
[[4,7],[3,4],[4,8]]
Any suggestions?
thanks in advance.
If you are using the rjson package, then you need to concatenate them into a list:
library(rjson)
json1 <- fromJSON(file = "json1")
json2 <- fromJSON(file = "json2")
json3 <- fromJSON(file = "json3")
jsonl <- list(json1, json2, json3)
jsonc <- toJSON(jsonc)
jsonc
[1] "[[[1,5],[5,7],[8,10]],[[5,6],[4,5],[5,8]],[[4,7],[3,4],[4,8]]]"
write(jsonc, file = "jsonc")
If you have many files, you can put them in a vector and use lapply to save some typing:
files <- c("json1", "json2", "json3")
jsonl <- lapply(files, function(f) fromJSON(file = f))
jsonc <- toJSON(jsonl)
write(jsonc, file = "jsonc")
I am pulling data directly from a Postgres database into R, where one of the columns in the Postgres table contains rows of JSON objects. I am trying to unpack the JSON objects and have them flatten into columns in an R dataframe, but so far, I'm getting mangled results.
Here's my code:
library(RPostgreSQL)
library(jsonlite)
drv <- dbDriver("PostgreSQL")
con <- dbConnect(drv, host="xxx", dbname="xxx", user="xxx", password="xxx")
query="select column1, column2, json from dummy_table limit 2"
resultSet <- dbSendQuery(con, query)
rawData<-fetch(resultSet,n=-1)
postgresqlCloseConnection(con)
rawData$json
[1]"{\"id\":{\"publisherName\":\"pub1\",\"visitorId\":\"visitor1\",\"timestamp\":1234},\"startAt\":4567,\"endAt\":8910}"
[2]"{\"id\":{\"publisherName\":\"pub2\",\"visitorId\":\"visitor2\",\"timestamp\":2345},\"startAt\":678,\"endAt\":91011}"
unpacked<-fromJSON(rawData$json, simplifyDataFrame=FALSE)
unpacked
$id
$id$publisherName
[1] "pub1"
$id$visitorId
[1] "visitor1"
$id$timestamp
[1] 1234
$startAt
[1] 4567
$endAt
[1] 8910
As you can see, it only unpacked the first JSON object, and it left things quasi-nested (which is fine, but optimally, i would want all the data to live in one level in a dataframe).
I would want the data to look like this:
unpacked
id.publisherName id.visitorId id.timestamp startAt endAt
pub1 visitor1 1234 4567 8910
pub2 visitor2 2345 678 91011
EDIT: Adding the rawData dataframe:
rawData<-structure(list(
column1 = c("abcd", "efgh"
),
column2 = structure(c(123, 456), class = c("POSIXct",
"POSIXt"), tzone = ""),
json = c("{\"id\":{\"publisherName\":\"pub1\",\"visitorId\":\"visitor1\",\"timestamp\":1234},\"startAt\":4567,\"endAt\":8910}",
"{\"id\":{\"publisherName\":\"pub2\",\"visitorId\":\"visitor2\",\"timestamp\":2345},\"startAt\":678,\"endAt\":91011}"
))
, .Names = c("column1", "column2", "json"),
row.names = 1:2, class = "data.frame")
Here's what happens with the paste function.
rawJSON <- paste("[", paste(rawData$json, collapse=","), "]")
rawJSON <- fromJSON(rawJSON, simplifyDataFrame=FALSE)
rawJSON
[[1]]
[[1]]$id
[[1]]$id$publisherName
[1] "pub1"
[[1]]$id$visitorId
[1] "visitor1"
[[1]]$id$timestamp
[1] 1234
[[1]]$startAt
[1] 4567
[[1]]$endAt
[1] 8910
[[2]]
[[2]]$id
[[2]]$id$publisherName
[1] "pub2"
[[2]]$id$visitorId
[1] "visitor2"
[[2]]$id$timestamp
[1] 2345
[[2]]$startAt
[1] 678
[[2]]$endAt
[1] 91011
The fromJSON function assumes that you are feeding it a single complete json string. Character vectors will be collapsed into single string. In your case your data contains multiple separate json objects. So you either need to convert them all individually:
lapply(rawData$json, fromJSON)
Or, to get the result that you're after, use stream_in to parse them as ndjson.
mydata <- jsonlite::stream_in(textConnection(rawData$json))
fromJSON(myjson)
See the jsonlite ?stream_in manual page for more details.
This question already has answers here:
Parse JSON with R
(6 answers)
Closed 2 years ago.
Is there a way to import data from a JSON file into R? More specifically, the file is an array of JSON objects with string fields, objects, and arrays. The RJSON Package isn't very clear on how to deal with this http://cran.r-project.org/web/packages/rjson/rjson.pdf.
First install the rjson package:
install.packages("rjson")
Then:
library("rjson")
json_file <- "http://api.worldbank.org/country?per_page=10®ion=OED&lendingtype=LNX&format=json"
json_data <- fromJSON(paste(readLines(json_file), collapse=""))
Update: since version 0.2.1
json_data <- fromJSON(file=json_file)
jsonlite will import the JSON into a data frame. It can optionally flatten nested objects. Nested arrays will be data frames.
> library(jsonlite)
> winners <- fromJSON("winners.json", flatten=TRUE)
> colnames(winners)
[1] "winner" "votes" "startPrice" "lastVote.timestamp" "lastVote.user.name" "lastVote.user.user_id"
> winners[,c("winner","startPrice","lastVote.user.name")]
winner startPrice lastVote.user.name
1 68694999 0 Lamur
> winners[,c("votes")]
[[1]]
ts user.name user.user_id
1 Thu Mar 25 03:13:01 UTC 2010 Lamur 68694999
2 Thu Mar 25 03:13:08 UTC 2010 Lamur 68694999
An alternative package is RJSONIO. To convert a nested list, lapply can help:
l <- fromJSON('[{"winner":"68694999", "votes":[
{"ts":"Thu Mar 25 03:13:01 UTC 2010", "user":{"name":"Lamur","user_id":"68694999"}},
{"ts":"Thu Mar 25 03:13:08 UTC 2010", "user":{"name":"Lamur","user_id":"68694999"}}],
"lastVote":{"timestamp":1269486788526,"user":
{"name":"Lamur","user_id":"68694999"}},"startPrice":0}]'
)
m <- lapply(
l[[1]]$votes,
function(x) c(x$user['name'], x$user['user_id'], x['ts'])
)
m <- do.call(rbind, m)
gives information on the votes in your example.
If the URL is https, like used for Amazon S3, then use getURL
json <- fromJSON(getURL('https://s3.amazonaws.com/bucket/my.json'))
First install the RJSONIO and RCurl package:
install.packages("RJSONIO")
install.packages("(RCurl")
Try below code using RJSONIO in console
library(RJSONIO)
library(RCurl)
json_file = getURL("https://raw.githubusercontent.com/isrini/SI_IS607/master/books.json")
json_file2 = RJSONIO::fromJSON(json_file)
head(json_file2)
load the packages:
library(httr)
library(jsonlite)
I have had issues converting json to dataframe/csv. For my case I did:
Token <- "245432532532"
source <- "http://......."
header_type <- "applcation/json"
full_token <- paste0("Bearer ", Token)
response <- GET(n_source, add_headers(Authorization = full_token, Accept = h_type), timeout(120), verbose())
text_json <- content(response, type = 'text', encoding = "UTF-8")
jfile <- fromJSON(text_json)
df <- as.data.frame(jfile)
then from df to csv.
In this format it should be easy to convert it to multiple .csvs if needed.
The important part is content function should have type = 'text'.
import httr package
library(httr)
Get the url
url <- "http://www.omdbapi.com/?apikey=72bc447a&t=Annie+Hall&y=&plot=short&r=json"
resp <- GET(url)
Print content of resp as text
content(resp, as = "text")
Print content of resp
content(resp)
Use content() to get the content of resp, but this time do not specify
a second argument. R figures out automatically that you're dealing
with a JSON, and converts the JSON to a named R list.