Nested JSON Dataframe - json

Question

I am trying to create a dataframe out of nested JSON files and am running into issues. So far, I've tried using tidyjson to stack the different columns like author, source, etc. However, when I get to "readers" (see the second JSON record), instead of column headers proceeding the ":" there is a unique id. I can't figure out how to get around that and create a dataframe. Here is the code I have so far:
art_df <- artJSON %>%
as.tbl_json %>%
gather_array %>%
enter_object("articles") %>% gather_array %>% # stack the articles
enter_object("author") %>% gather_array %>% # stack the authors
spread_values(articles.author = jstring("author")) %>% # extract item name and price
spread_values(articles.date = jstring("date")) %>%
) %>%
And the JSON file is structured like:
"articles" : {
"-2145189873" : {
"author" : "",
"date" : "",
"partialRecord" : true,
"source" : "reuters",
"text" : "",
"title" : "U.S. government loses to Russia's disinformation campaign: advisers | Reuters",
"url" : "reuters.com/article/us-usa-russia-disinformation-analysis-idUSKBN1492PA"
},
"-2143576379" : {
"author" : [ "", "", "CNN" ],
"date" : "Updated 8:21 AM ET, Mon June 19, 2017",
"readers" : {
"112851530401572877131" : true,
"115171523766335420644" : true
},
"source" : "cnn",
"text" : "Story highlightsThe higher the pressure, the more defiant the President becomesTrump continues to call the Russia probe a 'witch hunt' Washington (CNN)America is edging ever closer to a new long, national nightmare, given President Donald Trump's explosive response to signs that special counsel Robert Mueller is approaching the epicenter of White House power.The investigation has taken a serious turn in recent days, raising the clear possibility that whether Mueller finds wrongdoing or not, a period of political stress and upheaval is inevitable. Trump appears willing to test the bounds of convention and his own powers against a legal and political establishment that he believes is conspiring against him. The higher the pressure, the more defiant the President becomes -- even if it puts him at odds with his legal team.Trump has decided that \"this is a political fight and he's going to fight it,\" a White House official told CNN's Jeff Zeleny.But the sense of chaos around the White House is deepening. The President again called the Russia probe a \"witch hunt\" over the weekend while a legal adviser tried to walk back Trump's admission Friday that he is under investigation.Read More\"The MAKE AMERICA GREAT AGAIN agenda is doing very well despite the distraction of the Witch Hunt,\" Trump tweeted on Sunday.var id = '//platform.twitter.com/widgets.js'.replace(/\\s+/g, '');!!document.getElementById(id) || (function makeEmbedScript(d, id) (document, id));The MAKE AMERICA GREAT AGAIN agenda is doing very well despite the distraction of the Witch Hunt. Many new jobs, high business enthusiasm,..— Donald J. Trump (#realDonaldTrump) June 18, 2017The political stakes rose significantly in recent days amid signs the probe on Russian election hacking could eventually expand to consider Trump's actions.The President and his allies have declared open season on Mueller and Deputy Attorney General Rod Rosenstein, who is overseeing the probe. \"You are witnessing the single greatest WITCH HUNT in American political history - led by some very bad and conflicted people! #MAGA.\" Trump said in a tweet last week that signified an attempt to undermine both men./* global CNN, Modernizr */'use strict';CNN.INJECTOR.executeFeature('video').then(function () {CNN.VideoPlayer.handleUnmutePlayer = function handleUnmutePlayer(containerId, dataObj) {'use strict';var playerInstance,playerPropertyObj,rememberTime,unmuteCTA,unmuteIdSelector = 'unmute_' + containerId,isPlayerMute;dataObj = dataObj || {};if (CNN.VideoPlayer.getLibraryName(containerId) === 'fave') else isPlayerMute = (typeof dataObj.muted === 'boolean') ? dataObj.muted : false;if (CNN.VideoPlayer.playerProperties && CNN.VideoPlayer.playerProperties[containerId]) if (playerPropertyObj.mute && playerPropertyObj.contentPlayed) };CNN.VideoPlayer.showFlashSlate = function showFlashSlate(container) ;CNN.autoPlayVideoExist = (CNN.autoPlayVideoExist === true) ? true : false;var configObj = ,autoStartVideo = false,callbackObj,containerEl,currentVideoCollection = [],currentVideoCollectionId = '',isLivePlayer = false,moveToNextTimeout,mutePlayerEnabled = false,nextVideoId = '',nextVideoUrl = '',turnOnFlashMessaging = false,videoPinner,videoEndSlateImpl;if (CNN.autoPlayVideoExist === false) configObj.autostart = autoStartVideo;CNN.VideoPlayer.setPlayerProperties(configObj.markupId, autoStartVideo, isLivePlayer, mutePlayerEnabled);CNN.VideoPlayer.setFirstVideoInCollection(currentVideoCollection, configObj.markupId);videoEndSlateImpl = new CNN.VideoEndSlate('body-text_15');/*** Finds the next video ID and URL in the current collection, if available.* #param currentVideoId The video that is currently playing* #param containerId The parent container Id of the video element*/function findNextVideo(currentVideoId) findNextVideo('tv/2017/06/18/reliable-sources-journalists-trump-twitter-habit.cnn');/*** Navigates to the video leaf page of the next video in the current collection, if available.* #param currentVideoId The video that is currently playing* #param containerId The parent container Id of the video element*/function navigateToNextVideo(currentVideoId, containerId) callbackObj = ;if (typeof configObj.context !== 'string' || configObj.context.length 0) CNN.autoPlayVideoExist = (CNN.autoPlayVideoExist === true) ? true : false;CNN.VideoPlayer.getLibrary(configObj, callbackObj, isLivePlayer);});/* videodemanddust is a default feature of the injector */CNN.INJECTOR.scriptComplete('videodemanddust');JUST WATCHEDWhy journalists like Trump's Twitter habitReplayMore Videos ...MUST WATCHWhy journalists like Trump's Twitter habit 05:59The tweet sparked speculation that Trump may pull a nuclear option by firing Mueller and Rosenstein, a move senior aides have fervently advised against -- given that it could provoke a constitutional crisis.\"It looks like we have got a simmering tea kettle here, and the President seems to be testing the waters about whether he ought to go ahead and fire Robert Mueller,\" said Richard Ben-Veniste, a former Watergate prosecutor who is now a CNN legal analyst.\"You don't mess with the criminal justice system in this way ... the President is on very tenuous ground here,\" Ben-Veniste told CNN's Don Lemon Friday.Cuba to Trump: US in no 'condition to lecture us' on human rightsThe scorched-earth approach may have grave political consequences.Trump and his aides may well end up in the clear, but months of recriminations ahead will further polarize the capital and cast a shadow over his administration, which is already struggling for traction after a tumultuous first five months.If Mueller decides there is a case to answer for either the President or his aides in the issue of alleged collusion with Russia or over a potential cover-up, the political uproar will be incalculable.At best, it could further complicate Trump's efforts to pass a meaningful legislative agenda and ultimately his hopes of a successful presidency.At worst, it could force the nation to confront a third debate on whether to remove an elected President in just 45 years. The lessons of history suggest such traumas raise questions of political legitimacy and constitutional process that have the potential to sour life in Washington for years in the future./* global CNN, Modernizr */'use strict';CNN.INJECTOR.executeFeature('video').then(function () {CNN.VideoPlayer.handleUnmutePlayer = function handleUnmutePlayer(containerId, dataObj) {'use strict';var playerInstance,playerPropertyObj,rememberTime,unmuteCTA,unmuteIdSelector = 'unmute_' + containerId,isPlayerMute;dataObj = dataObj || {};if (CNN.VideoPlayer.getLibraryName(containerId) === 'fave') else isPlayerMute = (typeof dataObj.muted === 'boolean') ? dataObj.muted : false;if (CNN.VideoPlayer.playerProperties && CNN.VideoPlayer.playerProperties[containerId]) if (playerPropertyObj.mute && playerPropertyObj.contentPlayed) };CNN.VideoPlayer.showFlashSlate = function showFlashSlate(container) ;CNN.autoPlayVideoExist = (CNN.autoPlayVideoExist === true) ? true : false;var configObj = ,autoStartVideo = false,callbackObj,containerEl,currentVideoCollection = [],currentVideoCollectionId = '',isLivePlayer = false,moveToNextTimeout,mutePlayerEnabled = false,nextVideoId = '',nextVideoUrl = '',turnOnFlashMessaging = false,videoPinner,videoEndSlateImpl;if (CNN.autoPlayVideoExist === false) configObj.autostart = autoStartVideo;CNN.VideoPlayer.setPlayerProperties(configObj.markupId, autoStartVideo, isLivePlayer, mutePlayerEnabled);CNN.VideoPlayer.setFirstVideoInCollection(currentVideoCollection, configObj.markupId);videoEndSlateImpl = new CNN.VideoEndSlate('body-text_25');/*** Finds the next video ID and URL in the current collection, if available.* #param currentVideoId The video that is currently playing* #param containerId The parent container Id of the video element*/function findNextVideo(currentVideoId) findNextVideo('politics/2017/06/18/jay-sekulow-trump-obstruction-of-justice-sotu.cnn');/*** Navigates to the video leaf page of the next video in the current collection, if available.* #param currentVideoId The video that is currently playing* #param containerId The parent container Id of the video element*/function navigateToNextVideo(currentVideoId, containerId) callbackObj = ;if (typeof configObj.context !== 'string' || configObj.context.length 0) CNN.autoPlayVideoExist = (CNN.autoPlayVideoExist === true) ? true : false;CNN.VideoPlayer.getLibrary(configObj, callbackObj, isLivePlayer);});/* videodemanddust is a default feature of the injector */CNN.INJECTOR.scriptComplete('videodemanddust');JUST WATCHEDSekulow: POTUS has power to fire FBI directorReplayMore Videos ...MUST WATCHSekulow: POTUS has power to fire FBI director 00:10Trump's in 'a political fight and he's going to fight it'Back in the present, the President's mood is fueling the sense of crisis.\"One thing we've learned I think from the testimony of multiple people now is the President's pretty fired up about this, OK?\" Republican Sen. Marco Rubio said on NBC's \"Meet the Press\" Sunday.\"He, from every pronouncement we have seen, feels very strongly that he did nothing wrong, and he wants people to say that, because he feels very strongly about it,\" Rubio said. \"I don't think that's a mystery. And he's expressing himself in that way.\"But the more frustrated that Trump gets, the more he often hurts himself politically. A case in point is the firing of Comey and Trump's subsequent admission that he had Russia on his mind when he dismissed the FBI director, an admission that could provide a rationale for an obstruction of justice probe.\"Trump has a compulsion to counter-attack and is very pugnacious. I don't think it serves him well,\" former House Speaker Newt Gingrich, a Trump supporter, on ABC's \"This Week,\" adding that the President was right to be infuriated by \"this whole Russia baloney.\"/* global CNN, Modernizr */'use strict';CNN.INJECTOR.executeFeature('video').then(function () {CNN.VideoPlayer.handleUnmutePlayer = function handleUnmutePlayer(containerId, dataObj) {'use strict';var playerInstance,playerPropertyObj,rememberTime,unmuteCTA,unmuteIdSelector = 'unmute_' + containerId,isPlayerMute;dataObj = dataObj || {};if (CNN.VideoPlayer.getLibraryName(containerId) === 'fave') else isPlayerMute = (typeof dataObj.muted === 'boolean') ? dataObj.muted : false;if (CNN.VideoPlayer.playerProperties && CNN.VideoPlayer.playerProperties[containerId]) if (playerPropertyObj.mute && playerPropertyObj.contentPlayed) };CNN.VideoPlayer.showFlashSlate = function showFlashSlate(container) ;CNN.autoPlayVideoExist = (CNN.autoPlayVideoExist === true) ? true : false;var configObj = ,autoStartVideo = false,callbackObj,containerEl,currentVideoCollection = [],currentVideoCollectionId = '',isLivePlayer = false,moveToNextTimeout,mutePlayerEnabled = false,nextVideoId = '',nextVideoUrl = '',turnOnFlashMessaging = false,videoPinner,videoEndSlateImpl;if (CNN.autoPlayVideoExist === false) configObj.autostart = autoStartVideo;CNN.VideoPlayer.setPlayerProperties(configObj.markupId, autoStartVideo, isLivePlayer, mutePlayerEnabled);CNN.VideoPlayer.setFirstVideoInCollection(currentVideoCollection, configObj.markupId);videoEndSlateImpl = new CNN.VideoEndSlate('body-text_34');/*** Finds the next video ID and URL in the current collection, if available.* #param currentVideoId The video that is currently playing* #param containerId The parent container Id of the video element*/function findNextVideo(currentVideoId) findNextVideo('politics/2017/06/18/sotu-rubio-trump-talked-russia-probe-in-private.cnn');/*** Navigates to the video leaf page of the next video in the current collection, if available.* #param currentVideoId The video that is currently playing* #param containerId The parent container Id of the video element*/function navigateToNextVideo(currentVideoId, containerId) callbackObj = ;if (typeof configObj.context !== 'string' || configObj.context.length 0) CNN.autoPlayVideoExist = (CNN.autoPlayVideoExist === true) ? true : false;CNN.VideoPlayer.getLibrary(configObj, callbackObj, isLivePlayer);});/* videodemanddust is a default feature of the injector */CNN.INJECTOR.scriptComplete('videodemanddust');JUST WATCHEDRubio: Trump talked Russia probe in privateReplayMore Videos ...MUST WATCHRubio: Trump talked Russia probe in private 01:05Legal team contradicts the clientIn another sign of the gravity of the situation, Trump's legal team blitzed Sunday talk shows to contradict the President's tweet Friday where he said he is already under investigation.\"The President is not a subject or target of an investigation. That tweet was in response to a Washington Post story that ran with five unnamed sources, without identifying the agencies they represented, saying that the special counsel had broadened out his investigation to include the president,\" said Jay Sekulow, a Trump legal adviser on CNN's \"State of the Union.\"Sekulow also opened a conflict between the President's political team and his legal one, hinting that the President's tweets should be discounted even though the White House said they are an authentic expression of his views.\"It was 141 characters. There's a limitation on Twitter, as we all know,\" Sekulow said.Then, in a subsequent interview on \"Fox News Sunday,\" Sekulow sowed further confusion by saying Trump was under investigation -- then denied he said anything of the sort.Special counsel moving forwardLaw enforcement sources have told CNN that the special counsel is gathering information and considering whether there is evidence to launch a full scale obstruction investigation. Mueller's investigators have asked for information and will talk to Director of National Intelligence Dan Coats and National Security Agency Director Adm. Mike Rogers, according to a source familiar with the matter. Investigators have also sought information from recently retired NSA Deputy Richard Ledgett, according to the source. Ledgett wrote a memo, according to the source, documenting a conversation in which the president allegedly urged Rogers to help get the FBI to lift the cloud of the Russia investigation.Any confirmation that Mueller has expanded the investigation to consider whether the President obstructed justice with the Comey firing would carry no guarantee of a guilty finding.But it would deal a political blow to the White House in the battle of perception, since the idea that Trump is not under investigation has been at the centerpiece of his administration's defense strategy./* global CNN, Modernizr */'use strict';CNN.INJECTOR.executeFeature('video').then(function () {CNN.VideoPlayer.handleUnmutePlayer = function handleUnmutePlayer(containerId, dataObj) {'use strict';var playerInstance,playerPropertyObj,rememberTime,unmuteCTA,unmuteIdSelector = 'unmute_' + containerId,isPlayerMute;dataObj = dataObj || {};if (CNN.VideoPlayer.getLibraryName(containerId) === 'fave') else isPlayerMute = (typeof dataObj.muted === 'boolean') ? dataObj.muted : false;if (CNN.VideoPlayer.playerProperties && CNN.VideoPlayer.playerProperties[containerId]) if (playerPropertyObj.mute && playerPropertyObj.contentPlayed) };CNN.VideoPlayer.showFlashSlate = function showFlashSlate(container) ;CNN.autoPlayVideoExist = (CNN.autoPlayVideoExist === true) ? true : false;var configObj = ,autoStartVideo = false,callbackObj,containerEl,currentVideoCollection = [],currentVideoCollectionId = '',isLivePlayer = false,moveToNextTimeout,mutePlayerEnabled = false,nextVideoId = '',nextVideoUrl = '',turnOnFlashMessaging = false,videoPinner,videoEndSlateImpl;if (CNN.autoPlayVideoExist === false) configObj.autostart = autoStartVideo;CNN.VideoPlayer.setPlayerProperties(configObj.markupId, autoStartVideo, isLivePlayer, mutePlayerEnabled);CNN.VideoPlayer.setFirstVideoInCollection(currentVideoCollection, configObj.markupId);videoEndSlateImpl = new CNN.VideoEndSlate('body-text_48');/*** Finds the next video ID and URL in the current collection, if available.* #param currentVideoId The video that is currently playing* #param containerId The parent container Id of the video element*/function findNextVideo(currentVideoId) findNextVideo('tv/2017/06/18/inside-politics-6-18-notebooks.cnn');/*** Navigates to the video leaf page of the next video in the current collection, if available.* #param currentVideoId The video that is currently playing* #param containerId The parent container Id of the video element*/function navigateToNextVideo(currentVideoId, containerId) callbackObj = ;if (typeof configObj.context !== 'string' || configObj.context.length 0) CNN.autoPlayVideoExist = (CNN.autoPlayVideoExist === true) ? true : false;CNN.VideoPlayer.getLibrary(configObj, callbackObj, isLivePlayer);});/* videodemanddust is a default feature of the injector */CNN.INJECTOR.scriptComplete('videodemanddust');JUST WATCHEDTrump heads west to POTUS-friendly IowaReplayMore Videos ...MUST WATCHTrump heads west to POTUS-friendly Iowa 04:17",
"title" : "Trump defiant as pressure grows",
"url" : "cnn.com/2017/06/19/politics/trump-investigation/index.html"
},
Any help would be greatly appreciated. I've been stuck trying to read this file in for some time and cant figure it out. Thanks!!!

Answer 1

see https://rdrr.io/cran/jsonlite/man/flatten.html
data2 <- fromJSON("so.json", flatten = TRUE)