Error in fromJSON in R - json

I am trying to convert JSON file into data frame in R and had this error when I ran fromJSON in 'jsonlite' package.
Error in fromJSON(content, handler, default.size, depth, allowComments, :
Invalid JSON Node
I figured that the variable order are different in each row. In specific, for example, in the first row, "categories" is the second variable whereas it is the last variable in the third row (Please take a look at the data below). This is how the original JSON file is formatted, and I have several hundred thousand rows like this so can't manually change it.
Anyone can suggest how to fix this and successfully convert the JSON into data frame in R?
[{"asin": "0188399313", "categories": [["Baby"]], "description": "Wee-Go Glass baby bottles by LifeFactory (Babylife) are designed to grow with your child. The included clear cover can also serve as an easy to hold cup. Twist on the solid cap (sold separately) and use your bottles for storing juice or snacks. Perfect for a lunchbox or traveling. The bright colored silicone sleeve (patent pending) helps to protect the bottle from breakage and provides a great gripping surface and tactile experience during feeding. The bottle and sleeve can be boiled or put in the dishwasher together. They can also go in the freezer, making breast milk storage simple.", "title": "Lifefactory 4oz BPA Free Glass Baby Bottles - 4-pack-raspberry and Lilac", "price": 69.99, "imUrl": "http://ecx.images-amazon.com/images/I/41SwthpdD9L._SX300_.jpg", "brand": "Lifefactory", "related": {"also_bought": ["B002SG7K7A", "B003CJSXW8", "B004PW4186", "B002O3JH9Q", "B002O3NLIO", "B004HGSU28"], "also_viewed": ["B003CJSXW8", "B0052QOL1Q", "B004PW4186", "B00EN0OLZ8", "B00EN0OOQY", "B0049YS46K", "B00E64CBLM", "B00F9YOOS6", "B00AH9RPVQ", "B00BCU2R7G", "B002O3NLIO", "B008NZ4X2K", "B005NIDFEW", "B00DKPJCH4", "B00CZNGWWK", "B00DAKJIQ4", "B005CT55IQ", "B0049YRJM0", "B0071IEWD0", "B00E64CA68", "B00IUB3SKK", "B00A7AA6XY", "B001F50FFE", "B002HU9EO4", "B007HP11SQ", "B009WPUMX4", "B002O3JH9Q", "B00F2FT3K6", "B00I5CR35A", "B00BCTY5EK", "B002SG7K7A", "B00F2FLU2U", "B0062ZK0GQ", "B002UOFR66", "B0055LKQQ2", "B00A0FGN8I", "B00HMYCG2W", "B00DHFLUO0", "B0040HMPA2", "B00I5CT9XE", "B008B5MMNO", "B00BQYVNGO", "B00925WM28", "B00BGKC3EY", "B005Q3LSDO", "B0038JDVCE", "B0045I6IA4"], "bought_together": ["B002SG7K7A", "B003CJSXW8"], "buy_after_viewing": ["B003CJSXW8", "B0052QOL1Q", "B004PW4186", "B002SG7K7A"]},},
{"asin": "0188399518", "categories": [["Baby"]], "description": "The Planet Wise Flannel Wipes are 10 super soft, color coordinated, high-quality wipes that fit perfectly into the Planet Wise Wipe PouchTM.", "title": "Planetwise Flannel Wipes", "price": 15.95, "imUrl": "http://ecx.images-amazon.com/images/I/41otjnA4OGL._SY300_.jpg", "brand": "Planet Wise", "related": {"also_bought": ["B00G96N3YY", "B003XSEV2O", "B000138GNY", "B005WWIE3G", "B005WWI0DA", "B005WWIMGA", "B00DS4WPNK", "B0039VCRPI", "B001QIN6ME", "B004GMGLN8", "B00CJ2OWUG", "B004BDNJW8", "B003N0JXSO", "B003X3R6TO", "B002T5Q01C", "B006J2U4T0", "B00JBJDEC2", "B00380LVLG", "B00GJVM2NW", "B003AJHDQW", "B0043EDGP0", "B0012IJBUE", "B001OI0YWG", "B005DL5970", "B00305GSKS", "B00CQ9UUK8", "B002Y27PQ4", "B00AH8J448", "B00A3JXVZY", "B00483GAJU", "B001J6O6B8", "B0019ID6G2", "B005DL7LGM", "B00APSYDXM", "B004D5KJJA", "B0021HR94K", "0757302661", "B001HX4DNE"], "also_viewed": ["B000138GNY", "B00G96N3YY", "B006J2U4T0", "B00GLBR3C0", "B0039VCRPI", "B0012IJBUE", "B001IA2XGK", "B001R2L6PS", "B003X3R6TO", "B00DS4WPNK", "B005WWIE3G", "B004BDNJW8", "B005WWIMGA", "B005WWI0DA", "B003XSEV2O", "B004A5VGPE", "B004A5TTGC", "B00JBJDEC2", "B004GMGLN8", "B009W6PM9M", "B004RGL9I0", "B0040IWMTK", "B00J3C0XC6", "B00HO1HBC8", "B00G96N18M", "B00A3V0XDA", "B0032AMM9M", "B001QIN6ME", "B00A3V0XEY", "B00CIUAB40", "B00J3C0Y1G", "B005IPE0KI", "B00L0W5B7Y", "B00FZGLKQW", "B00HBYOE3W", "B004A5TT4O", "B00B6KS578", "B002HOQOMA", "B004G792KW", "B002UD6BZS", "B002XVRKHK", "B002HOQOUW", "B002UD6C1Q", "B00APSYDXM", "B00H07AZ42", "B002LZX2BG", "B00BFQVZNO", "B00EZKNVYS", "B00DCLYYJW", "B00AHG8H1Q", "B002QZ64T8", "B003N0JXSO", "B003AJHDQW", "B00DN9NW32", "B00CHPJ0I4", "B005WXMJUY", "B00D4LFC3W"], "bought_together": ["B00G96N3YY", "B003XSEV2O"], "buy_after_viewing": ["B000138GNY", "B006J2U4T0", "B00G96N3YY", "B005WWIMGA"]},},
{"asin": "0188399399", "description": "The Planet Wise Wipe PouchTM features our patent pending no-leak design so your wipes will stay moist and not wick into your diaper bag. Features a unique snap down design to eliminate wicking from the zipper area. Most standard size wipes will fit into our Wipe PouchTM perfectly! Pouches are made of the same high quality fabrics as our Wet Bags, this pouch will last and last. Pouches are anti-microbial too! After you are done with the baby years, keep this pouch to use for just about anything you need to transport: make-up, toiletries, anything. Made in the USA!", "title": "Planetwise Wipe Pouch", "price": 10.95, "imUrl": "http://ecx.images-amazon.com/images/I/61x8h9u6mxL._SY300_.jpg", "related": {"also_bought": ["B005WWI0DA", "B005WWIMGA", "B006J2U4T0", "B000138GNY", "B003XSEV2O", "B005WWIE3G", "B00DS4WPNK", "B0039VCRPI", "B003GSLCOG", "B0018B15FE", "B002MN3JY2"], "also_viewed": ["B005WWIMGA", "B00G96N3YY", "B00DAI76TC", "B0067GKHVS", "B005WWIE3G", "B004RGL9I0", "B000138GNY", "B00JBJDEC2", "B00CMCJ2AS", "B005WWI0DA", "B002UD6C16", "B00E401FVU", "B004GMGLN8", "B0012IJBUE", "B00BFQVZNO", "B006J2U4T0", "B000VV21G4", "B002HOQOUW", "B003X3R6TO", "B000GZJIVQ", "B0052AIF00", "B005GQ6606", "B003TZHCA4", "B001QIN6ME", "B003XSEV2O", "B00GLBR3C0", "B00DW9R03G", "B00J3C0XC6", "0188399542", "B00DQL6CIE", "B002UD6C1G", "B00A3V0XDA", "B0039VCRPI", "B00EZKNVYS", "B006QRFDHQ", "B00DS4WPNK", "B00CQ9UUK8", "B0087A2PKS", "B00GGNICSW", "B0037NXP18", "B000YCNLSC", "B00FJG945W", "B00FZGLKQW", "B004A5VGPE", "B00D4LFC3W", "B00G96N18M", "B00439UNR4", "B004A5TT4O", "B005DL7LGM", "B006SD2JUW", "B006Z6HXYY", "B006QRFDZI", "B00I0P7VYK", "B00AZWDM0I", "B000ZKHVMU", "B007799PGC", "B00H58UNMU", "B00JKUU3TE"], "buy_after_viewing": ["B005WWIMGA", "B00G96N3YY", "B00DAI76TC", "B0067GKHVS"]},, "categories": [["Baby"]]},
{"asin": "0316967297", "description": "Hand crafted set includes 1 full quilt (76x86 inches) and 2 standard shams (20x26 inches). Face cloth and fill are 100 natural cotton. Prewashed for out of the bag comfort. Hand crafted with embroidery. Machine washable. Made in China", "title": "Annas Dream Full Quilt with 2 Shams", "price": 109.95, "imUrl": "http://ecx.images-amazon.com/images/I/51%2BZ1%2BNeukL._SY300_.jpg", "related": {"also_viewed": ["B009LTER3W", "B00575TI5Q", "B004NSYYJI"], "buy_after_viewing": ["B009LTER3W", "B001MX5EE6", "B00575TI5Q", "B0029009TG"]},, "categories": [["Baby"]]},
{"asin": "0615447279", "categories": [["Baby"]], "description": "Thumbuddy To Love- The Binky Fairy helps children give up pacifier sucking without the fuss and tears. The adorable book comes with a matching Binky Fairy puppet and just like the Tooth Fairy, the book is read the night before so children understand where thier pacifiers will go. They awake to find an adorable Binky Fairy puppet under thier pillow knowing that the Binky Fairy came! Each book comes with a success chart and stickers. Winner of the PTPA Awards (Parent Tested, Parent Approved). Recommended for ages 2-4.", "title": "Stop Pacifier Sucking without tears with Thumbuddy To Love s Binky Fairy Puppet and Adorable Book", "price": 16.95, "imUrl": "http://ecx.images-amazon.com/images/I/51RKKENlq%2BL._SY300_.jpg", "brand": "", "related": {"also_bought": ["0979670004", "1601310234", "B005G172KE", "1575422573", "1905417896", "B0044D0HA2", "B001GQ2CPI"], "also_viewed": ["0979670004", "1575422573", "1493535943", "0615273645", "1601310234", "0992616727", "0615273793", "0763623644", "1907152962", "0992616719", "B0016ZX7AI", "0698400488", "1608442713", "0451416058", "1581176848", "B0072CTECY", "B005G172KE", "B0016P8K3W", "0375822704", "0979201004", "0961478020", "160131048X", "1572245859"], "bought_together": ["B005G172KE"], "buy_after_viewing": ["0979670004", "1575422573", "B005G172KE", "1493535943"]},}]

Related

Concatenate values from non-adjacent objects based on multiple matching criteria

I received help on a related question previously on this forum and am wondering if there is a similarly straightforward way to resolve a more complex issue.
Given the following snippet, is there a way to merge the partial sentence (the one which does not end with a "[punctuation mark][white space]" pattern) with its remainder based on the matching TextSize? When I tried to adjust the answer from the related question I quickly ran into issues, but I am basically looking to translate a rule such as if .Text !endswith("[punctuation mark][white space]") then .Text + next .Text where .TextSize matches
{
"Text": "Was it political will that established social democratic policies in the 1930s and ",
"Path": "P",
"TextSize": 9
},
{
"Text": "31 Lawrence Mishel and Jessica Schieder, Economic Policy Institute website, May 24, 2016 at (https://www.epi.org/publication/as-union-membership-has-fallen-the-top-10-percent-have-been-getting-a-larger-share-of-income/). ",
"Path": "Footnote",
"TextSize": 8
},
{
"Text": "Fig. 9.2 Higher union membership has been associated with a higher share of income to lower income brackets (the lower 90%) and a lower share of income to the top 10% of earners. ",
"Path": "P",
"TextSize": 8
},
{
"Text": "1940s, or that undermined them after the 1970s? Or was it abundant and cheap energy resources that enabled social democratic policies to work until the 1970s, and energy constraints that forced a restructuring of policy after the 1970s? ",
"Path": "P",
"TextSize": 9
},
{
"Text": "Recall that my economic modeling discussed in Chap. 6 shows that, even with no change in the assumption related to labor \u201cbargaining power,\u201d you can explain a shift from increasing to declining income equality (higher equality expressed as a higher wage share) by a corresponding shift from a period of rapidly increasing per capita resource consumption to one of constant per capita resource consumption. ",
"Path": "P",
"TextSize": 9
}
The result I'm looking for would be as follows:
{
"Text": "Was it political will that established social democratic policies in the 1930s and 1940s, or that undermined them after the 1970s? Or was it abundant and cheap energy resources that enabled social democratic policies to work until the 1970s, and energy constraints that forced a restructuring of policy after the 1970s? ",
"Path": "P",
"TextSize": 9
},
{
"Text": "31 Lawrence Mishel and Jessica Schieder, Economic Policy Institute website, May 24, 2016 at (https://www.epi.org/publication/as-union-membership-has-fallen-the-top-10-percent-have-been-getting-a-larger-share-of-income/). ",
"Path": "Footnote",
"TextSize": 8
},
{
"Text": "Fig. 9.2 Higher union membership has been associated with a higher share of income to lower income brackets (the lower 90%) and a lower share of income to the top 10% of earners. ",
"Path": "P",
"TextSize": 8
},
{
"Text": "Recall that my economic modeling discussed in Chap. 6 shows that, even with no change in the assumption related to labor \u201cbargaining power,\u201d you can explain a shift from increasing to declining income equality (higher equality expressed as a higher wage share) by a corresponding shift from a period of rapidly increasing per capita resource consumption to one of constant per capita resource consumption. ",
"Path": "P",
"TextSize": 9
}

The following, which assumes the input is a valid JSON array, will merge every .Text with at most one successor, but can easily be modified to merge multiple .Text values together as shown in Part 2 below.
Part 1
# input and output: an array of {Text, Path, TextSize} objects.
# Attempt to merge the .Text of the $i-th object with the .Text of a subsequent compatible object.
# If a merge is successful, the subsequent object is removed.
def attempt_to_merge_next($i):
.[$i].TextSize as $class
| first( (range($i+1; length) as $j | select(.[$j].TextSize == $class) | $j) // null) as $j
| if $j then .[$i].Text += .[$j].Text | del(.[$j])
else .
end;
reduce range(0; length) as $i (.;
if .[$i] == null then .
elif .[$i].Text|test("[,.?:;]\\s*$")|not
then attempt_to_merge_next($i)
else .
end)
Part 2
Using the above def:
def merge:
def m($i):
if $i >= length then .
elif .[$i].Text|test("[,.?:;]\\s*$")|not
then attempt_to_merge_next($i) as $x
| if ($x|length) == length then m($i+1)
else $x|m($i)
end
else m($i+1)
end ;
m(0);
merge

Altering JSON Structure

So I am using a webscraper to pull information on sneakers from a website. The son data that comes back is structured like so
[
{
"web-scraper-order": "1554084909-97",
"web-scraper-start-url": "https://www.goat.com/sneakers",
"productlink": "$200AIR JORDAN 6 RETRO 'INFRARED' 2019",
"productlink-href": "https://www.goat.com/sneakers/air-jordan-6-retro-black-infrared-384664-060",
"name": "Air Jordan 6 Retro 'Infrared' 2019",
"price": "Buy New - $200",
"description": "The 2019 edition of the Air Jordan 6 Retro ‘Infrared’ is true to the original colorway, which Michael Jordan wore when he captured his first NBA title. Dressed primarily in black nubuck with a reflective 3M layer underneath, the mid-top features Infrared accents on the midsole, heel tab and lace lock. Nike Air branding adorns the heel and sockliner, an OG detail last seen on the 2000 retro.",
"releasedate": "2019-02-16",
"colorway": "Black/Infrared 23-Black",
"brand": "Air Jordan",
"designer": "Tinker Hatfield",
"technology": "Air",
"maincolor": "Black",
"silhouette": "Air Jordan 6",
"nickname": "Infrared",
"category": "lifestyle",
"image-src": "https://image.goat.com/crop/1250/attachments/product_template_additional_pictures/images/018/675/318/original/464372_01.jpg.jpeg"
},
{
"web-scraper-order": "1554084922-147",
"web-scraper-start-url": "https://www.goat.com/sneakers",
"productlink": "$190YEEZY BOOST 350 V2 'CREAM WHITE / TRIPLE WHITE'",
"productlink-href": "https://www.goat.com/sneakers/yeezy-boost-350-v2-cream-white-cp9366",
"name": "Yeezy Boost 350 V2 'Cream White / Triple White'",
"price": "Buy New - $220",
"description": "First released on April 29, 2017, the Yeezy Boost 350 V2 ‘Cream White’ combines a cream Primeknit upper with tonal cream SPLY 350 branding, and a translucent white midsole housing full-length Boost. Released again in October 2018, this retro helped fulfill Kanye West’s oft-repeated ‘YEEZYs for everyone’ Twitter mantra, as adidas organized the biggest drop in Yeezy history by promising pre-sale to anyone who signed up on the website. Similar to the first release, the ‘Triple White’ 2018 model features a Primeknit upper, a Boost midsole and custom adidas and Yeezy co-branding on the insole.",
"releasedate": "2017-04-29",
"colorway": "Cream White/Cream White/Core White",
"brand": "adidas",
"designer": "Kanye West",
"technology": "Boost",
"maincolor": "White",
"silhouette": "Yeezy Boost 350",
"nickname": "Cream White / Triple White",
"category": "lifestyle",
"image-src": "https://image.goat.com/crop/1250/attachments/product_template_additional_pictures/images/014/822/695/original/116662_03.jpg.jpeg"
},
However, I want to change it so that the top level node is sneakers and the next level down would be a specific sneaker brand ( Jordan, Nike, Adidas) and then the list of sneakers that belong to that brand. So my JSON structure would look something like this
Sneakers {
Adidas :{
[shoe1,
shoe2,
....
] },
Jordan: {
[shoe1,
shoe2,
....
]
}
}
I am not sure what tool I could use to do that. Any help would be greatly appreciated. All I have at the moment is the JSON file and it is not in the structure that I want it to be in.

One way of doing this would be to populate a dict whose keys are brand names and their values are lists of sneaker records. Assuming that data is your original list, here's the code:
sneakers_by_brand = {}
for record in data:
if sneakers_by_brand.get(record.get("brand")):
sneakers_by_brand[record.get("brand")].append(record)
else:
sneakers_by_brand[record.get("brand")] = [record]
print(sneakers_by_brand)

How can I sort JSON object?

I have been looking for this for a while now. I want to sort the JSON file following way:
{
"rss": {
"-version": "2.0",
"channel": {
"title": "pubmed: wonpil im",
"link": "https://www.ncbi.nlm.nih.gov/sites/entrez?cmd=Search&db=PubMed&term=wonpil%20im",
"description": "NCBI: db=pubmed; Term=wonpil im",
"language": "en-us",
"docs": "http://blogs.law.harvard.edu/tech/rss",
"ttl": "1440",
"image": {
"title": "NCBI pubmed",
"url": "https://www.ncbi.nlm.nih.gov/entrez/query/static/gifs/iconsml.gif",
"link": "https://www.ncbi.nlm.nih.gov/sites/entrez",
"description": "PubMed comprises more than millions of citations for biomedical literature from MEDLINE, life science journals, and online books. Citations may include links to full-text content from PubMed Central and publisher web sites."
},
"item": [
{
"title": "Modeling and simulation of bacterial outer membranes and interactions with membrane proteins.",
"link": "https://www.ncbi.nlm.nih.gov/pubmed/28157627?dopt=Abstract",
"description": "
<table border=\"0\" width=\"100%\"><tr><td align=\"left\"><img src=\"//www.ncbi.nlm.nih.gov/corehtml/query/egifs/http:--linkinghub.elsevier.com-ihub-images-PubMedLink.gif\" border=\"0\"/> </td><td align=\"right\">Related Articles</td></tr></table>
<p><b>Modeling and simulation of bacterial outer membranes and interactions with membrane proteins.</b></p>
<p>Curr Opin Struct Biol. 2017 Jan 31;43:131-140</p>
<p>Authors: Patel DS, Qi Y, Im W</p>
<p>Abstract<br/>
The outer membrane (OM) of Gram-negative bacteria is composed of phospholipids in the periplasmic leaflet and lipopolysaccharides (LPS) in the external leaflet, along with β-barrel OM proteins (OMPs) and lipidated periplasmic lipoproteins. As a defensive barrier to toxic compounds, an LPS molecule has high antigenic diversity and unique combination of OM-anchored lipid A with core oligosaccharides and O-antigen polysaccharides, creating dynamic protein-LPS and LPS-LPS interactions. Here, we review recent efforts on modeling and simulation of native-like bacterial OMs to explore structures, dynamics, and interactions of different OM components and their roles in transportation of ions, substrates, and antibiotics across the OM and accessibility of monoclonal antibodies (mAbs) to surface epitopes. Simulation studies attempting to provide insight into the structural basis for LPS transport and OMP insertion in the bacterial OM are also highlighted.<br/>
</p><p>PMID: 28157627 [PubMed - as supplied by publisher]</p>
",
"author": " Patel DS, Qi Y, Im W",
"category": "Curr Opin Struct Biol",
"guid": {
"-isPermaLink": "false",
"#text": "PubMed:28157627"
}
},
{
"title": "Refinement of OprH-LPS Interactions by Molecular Simulations.",
"link": "https://www.ncbi.nlm.nih.gov/pubmed/28122220?dopt=Abstract",
"description": "
<table border=\"0\" width=\"100%\"><tr><td align=\"left\"><img src=\"//www.ncbi.nlm.nih.gov/corehtml/query/egifs/http:--linkinghub.elsevier.com-ihub-images-cellhub.gif\" border=\"0\"/> </td><td align=\"right\">Related Articles</td></tr></table>
<p><b>Refinement of OprH-LPS Interactions by Molecular Simulations.</b></p>
<p>Biophys J. 2017 Jan 24;112(2):346-355</p>
<p>Authors: Lee J, Patel DS, Kucharska I, Tamm LK, Im W</p>
<p>Abstract<br/>
The outer membrane (OM) of Gram-negative bacteria is composed of lipopolysaccharide (LPS) in the outer leaflet and phospholipids in the inner leaflet. The outer membrane protein H (OprH) of Pseudomonas aeruginosa provides an increased stability to the OMs by directly interacting with LPS. Here we report the influence of various P. aeruginosa and, for comparison, Escherichia coli LPS environments on the physical properties of the OMs and OprH using all-atom molecular dynamics simulations. The simulations reveal that although the P. aeruginosa OMs are thinner hydrophobic bilayers than the E. coli OMs, which is expected from the difference in the acyl chain length of their lipid A, this effect is almost imperceptible around OprH due to a dynamically adjusted hydrophobic match between OprH and the OM. The structure and dynamics of the extracellular loops of OprH show distinct behaviors in different LPS environments. Including the O-antigen greatly reduces the flexibility of the OprH loops and increases the interactions between these loops and LPS. Furthermore, our study shows that the interactions between OprH and LPS mainly depend on the secondary structure of OprH and the chemical structure of LPS, resulting in distinctive patterns in different LPS environments.<br/>
</p><p>PMID: 28122220 [PubMed - in process]</p>
",
"author": " Lee J, Patel DS, Kucharska I, Tamm LK, Im W",
"category": "Biophys J",
"guid": {
"-isPermaLink": "false",
"#text": "PubMed:28122220"
}
},
{
"title": "CHARMM-GUI MDFF/xMDFF Utilizer for Molecular Dynamics Flexible Fitting Simulations in Various Environments.",
"link": "https://www.ncbi.nlm.nih.gov/pubmed/27936734?dopt=Abstract",
"description": "
<table border=\"0\" width=\"100%\"><tr><td align=\"left\"><img src=\"//www.ncbi.nlm.nih.gov/corehtml/query/egifs/http:--pubs.acs.org-images-pubmed-acspubs.jpg\" border=\"0\"/> </td><td align=\"right\">Related Articles</td></tr></table>
<p><b>CHARMM-GUI MDFF/xMDFF Utilizer for Molecular Dynamics Flexible Fitting Simulations in Various Environments.</b></p>
<p>J Phys Chem B. 2016 Dec 23;:</p>
<p>Authors: Qi Y, Lee J, Singharoy A, McGreevy R, Schulten K, Im W</p>
<p>Abstract<br/>
X-ray crystallography and cryo-electron microscopy are two popular methods for the structure determination of biological molecules. Atomic structures are derived through the fitting and refinement of an initial model into electron density maps constructed by both experiments. Two computational approaches, MDFF and xMDFF, have been developed to facilitate this process by integrating the experimental data with molecular dynamics simulation. However, the setup of an MDFF/xMDFF simulation requires knowledge of both experimental and computational methods, which is not straightforward for nonexpert users. In addition, sometimes it is desirable to include realistic environments, such as explicit solvent and lipid bilayers during the simulation, which poses another challenge even for expert users. To alleviate these difficulties, we have developed MDFF/xMDFF Utilizer in CHARMM-GUI that helps users to set up an MDFF/xMDFF simulation. The capability of MDFF/xMDFF Utilizer is greatly enhanced by integration with other CHARMM-GUI modules, including protein structure manipulation, a diverse set of lipid types, and all-atom CHARMM and coarse-grained PACE force fields. With this integration, various simulation environments are available for MDFF Utilizer (vacuum, implicit/explicit solvent, and bilayers) and xMDFF Utilizer (vacuum and solution). In this work, three examples are shown to demonstrate the usage of MDFF/xMDFF Utilizer.<br/>
</p><p>PMID: 27936734 [PubMed - as supplied by publisher]</p>
",
"author": " Qi Y, Lee J, Singharoy A, McGreevy R, Schulten K, Im W",
"category": "J Phys Chem B",
"guid": {
"-isPermaLink": "false",
"#text": "PubMed:27936734"
}
},
]
}
}
}
I want this to be sorted according to "PubMed" numbers following way:
{
"rss": {
"-version": "2.0",
"channel": {
"title": "pubmed: wonpil im",
"link": "https://www.ncbi.nlm.nih.gov/sites/entrez?cmd=Search&db=PubMed&term=wonpil%20im",
"description": "NCBI: db=pubmed; Term=wonpil im",
"language": "en-us",
"docs": "http://blogs.law.harvard.edu/tech/rss",
"ttl": "1440",
"image": {
"title": "NCBI pubmed",
"url": "https://www.ncbi.nlm.nih.gov/entrez/query/static/gifs/iconsml.gif",
"link": "https://www.ncbi.nlm.nih.gov/sites/entrez",
"description": "PubMed comprises more than millions of citations for biomedical literature from MEDLINE, life science journals, and online books. Citations may include links to full-text content from PubMed Central and publisher web sites."
},
"item": [
{
"title": "CHARMM-GUI MDFF/xMDFF Utilizer for Molecular Dynamics Flexible Fitting Simulations in Various Environments.",
"link": "https://www.ncbi.nlm.nih.gov/pubmed/27936734?dopt=Abstract",
"description": "
<table border=\"0\" width=\"100%\"><tr><td align=\"left\"><img src=\"//www.ncbi.nlm.nih.gov/corehtml/query/egifs/http:--pubs.acs.org-images-pubmed-acspubs.jpg\" border=\"0\"/> </td><td align=\"right\">Related Articles</td></tr></table>
<p><b>CHARMM-GUI MDFF/xMDFF Utilizer for Molecular Dynamics Flexible Fitting Simulations in Various Environments.</b></p>
<p>J Phys Chem B. 2016 Dec 23;:</p>
<p>Authors: Qi Y, Lee J, Singharoy A, McGreevy R, Schulten K, Im W</p>
<p>Abstract<br/>
X-ray crystallography and cryo-electron microscopy are two popular methods for the structure determination of biological molecules. Atomic structures are derived through the fitting and refinement of an initial model into electron density maps constructed by both experiments. Two computational approaches, MDFF and xMDFF, have been developed to facilitate this process by integrating the experimental data with molecular dynamics simulation. However, the setup of an MDFF/xMDFF simulation requires knowledge of both experimental and computational methods, which is not straightforward for nonexpert users. In addition, sometimes it is desirable to include realistic environments, such as explicit solvent and lipid bilayers during the simulation, which poses another challenge even for expert users. To alleviate these difficulties, we have developed MDFF/xMDFF Utilizer in CHARMM-GUI that helps users to set up an MDFF/xMDFF simulation. The capability of MDFF/xMDFF Utilizer is greatly enhanced by integration with other CHARMM-GUI modules, including protein structure manipulation, a diverse set of lipid types, and all-atom CHARMM and coarse-grained PACE force fields. With this integration, various simulation environments are available for MDFF Utilizer (vacuum, implicit/explicit solvent, and bilayers) and xMDFF Utilizer (vacuum and solution). In this work, three examples are shown to demonstrate the usage of MDFF/xMDFF Utilizer.<br/>
</p><p>PMID: 27936734 [PubMed - as supplied by publisher]</p>
",
"author": " Qi Y, Lee J, Singharoy A, McGreevy R, Schulten K, Im W",
"category": "J Phys Chem B",
"guid": {
"-isPermaLink": "false",
"#text": "PubMed:27936734"
}
},
{
"title": "Modeling and simulation of bacterial outer membranes and interactions with membrane proteins.",
"link": "https://www.ncbi.nlm.nih.gov/pubmed/28157627?dopt=Abstract",
"description": "
<table border=\"0\" width=\"100%\"><tr><td align=\"left\"><img src=\"//www.ncbi.nlm.nih.gov/corehtml/query/egifs/http:--linkinghub.elsevier.com-ihub-images-PubMedLink.gif\" border=\"0\"/> </td><td align=\"right\">Related Articles</td></tr></table>
<p><b>Modeling and simulation of bacterial outer membranes and interactions with membrane proteins.</b></p>
<p>Curr Opin Struct Biol. 2017 Jan 31;43:131-140</p>
<p>Authors: Patel DS, Qi Y, Im W</p>
<p>Abstract<br/>
The outer membrane (OM) of Gram-negative bacteria is composed of phospholipids in the periplasmic leaflet and lipopolysaccharides (LPS) in the external leaflet, along with β-barrel OM proteins (OMPs) and lipidated periplasmic lipoproteins. As a defensive barrier to toxic compounds, an LPS molecule has high antigenic diversity and unique combination of OM-anchored lipid A with core oligosaccharides and O-antigen polysaccharides, creating dynamic protein-LPS and LPS-LPS interactions. Here, we review recent efforts on modeling and simulation of native-like bacterial OMs to explore structures, dynamics, and interactions of different OM components and their roles in transportation of ions, substrates, and antibiotics across the OM and accessibility of monoclonal antibodies (mAbs) to surface epitopes. Simulation studies attempting to provide insight into the structural basis for LPS transport and OMP insertion in the bacterial OM are also highlighted.<br/>
</p><p>PMID: 28157627 [PubMed - as supplied by publisher]</p>
",
"author": " Patel DS, Qi Y, Im W",
"category": "Curr Opin Struct Biol",
"guid": {
"-isPermaLink": "false",
"#text": "PubMed:28157627"
}
},
{
"title": "Refinement of OprH-LPS Interactions by Molecular Simulations.",
"link": "https://www.ncbi.nlm.nih.gov/pubmed/28122220?dopt=Abstract",
"description": "
<table border=\"0\" width=\"100%\"><tr><td align=\"left\"><img src=\"//www.ncbi.nlm.nih.gov/corehtml/query/egifs/http:--linkinghub.elsevier.com-ihub-images-cellhub.gif\" border=\"0\"/> </td><td align=\"right\">Related Articles</td></tr></table>
<p><b>Refinement of OprH-LPS Interactions by Molecular Simulations.</b></p>
<p>Biophys J. 2017 Jan 24;112(2):346-355</p>
<p>Authors: Lee J, Patel DS, Kucharska I, Tamm LK, Im W</p>
<p>Abstract<br/>
The outer membrane (OM) of Gram-negative bacteria is composed of lipopolysaccharide (LPS) in the outer leaflet and phospholipids in the inner leaflet. The outer membrane protein H (OprH) of Pseudomonas aeruginosa provides an increased stability to the OMs by directly interacting with LPS. Here we report the influence of various P. aeruginosa and, for comparison, Escherichia coli LPS environments on the physical properties of the OMs and OprH using all-atom molecular dynamics simulations. The simulations reveal that although the P. aeruginosa OMs are thinner hydrophobic bilayers than the E. coli OMs, which is expected from the difference in the acyl chain length of their lipid A, this effect is almost imperceptible around OprH due to a dynamically adjusted hydrophobic match between OprH and the OM. The structure and dynamics of the extracellular loops of OprH show distinct behaviors in different LPS environments. Including the O-antigen greatly reduces the flexibility of the OprH loops and increases the interactions between these loops and LPS. Furthermore, our study shows that the interactions between OprH and LPS mainly depend on the secondary structure of OprH and the chemical structure of LPS, resulting in distinctive patterns in different LPS environments.<br/>
</p><p>PMID: 28122220 [PubMed - in process]</p>
",
"author": " Lee J, Patel DS, Kucharska I, Tamm LK, Im W",
"category": "Biophys J",
"guid": {
"-isPermaLink": "false",
"#text": "PubMed:28122220"
}
},
]
}
}
}
Which is simply sorting according to given "#text": "PubMed:28122220". I would appreciate if anyone can help.

How to split very large JSON file in R?

I am currently working with Amazon User Metadata of size roughly around 9 GB. I have this data in .json format and want to read in R, due to such huge volume of data I want to divide the files in small chunks (either by size or by number of elements). If you can help me out how can I do this in R. The format of file is as follows:
{
"asin": "0000031852",
"title": "Girls Ballet Tutu Zebra Hot Pink",
"price": 3.17,
"imUrl": "http://ecx.images-amazon.com/images/I/51fAmVkTbyL._SY300_.jpg",
"related":
{
"also_bought": ["B00JHONN1S", "B002BZX8Z6", "B00D2K1M3O", "0000031909", "B00613WDTQ", "B00D0WDS9A", "B00D0GCI8S", "0000031895", "B003AVKOP2", "B003AVEU6G", "B003IEDM9Q", "B002R0FA24", "B00D23MC6W", "B00D2K0PA0", "B00538F5OK", "B00CEV86I6", "B002R0FABA", "B00D10CLVW", "B003AVNY6I", "B002GZGI4E", "B001T9NUFS", "B002R0F7FE", "B00E1YRI4C", "B008UBQZKU", "B00D103F8U", "B007R2RM8W"],
"also_viewed": ["B002BZX8Z6", "B00JHONN1S", "B008F0SU0Y", "B00D23MC6W", "B00AFDOPDA", "B00E1YRI4C", "B002GZGI4E", "B003AVKOP2", "B00D9C1WBM", "B00CEV8366", "B00CEUX0D8", "B0079ME3KU", "B00CEUWY8K", "B004FOEEHC", "0000031895", "B00BC4GY9Y", "B003XRKA7A", "B00K18LKX2", "B00EM7KAG6", "B00AMQ17JA", "B00D9C32NI", "B002C3Y6WG", "B00JLL4L5Y", "B003AVNY6I", "B008UBQZKU", "B00D0WDS9A", "B00613WDTQ", "B00538F5OK", "B005C4Y4F6", "B004LHZ1NY", "B00CPHX76U", "B00CEUWUZC", "B00IJVASUE", "B00GOR07RE", "B00J2GTM0W", "B00JHNSNSM", "B003IEDM9Q", "B00CYBU84G", "B008VV8NSQ", "B00CYBULSO", "B00I2UHSZA", "B005F50FXC", "B007LCQI3S", "B00DP68AVW", "B009RXWNSI", "B003AVEU6G", "B00HSOJB9M", "B00EHAGZNA", "B0046W9T8C", "B00E79VW6Q", "B00D10CLVW", "B00B0AVO54", "B00E95LC8Q", "B00GOR92SO", "B007ZN5Y56", "B00AL2569W", "B00B608000", "B008F0SMUC", "B00BFXLZ8M"],
"bought_together": ["B002BZX8Z6"]
},
"salesRank": {"Toys & Games": 211836},
"brand": "Coxlures",
"categories": [["Sports & Outdoors", "Other Sports", "Dance"]]
}

Why my angular.forEach loop isn't working

I am not sure what's going wrong, cause it looks pretty straight forward to me. I have few things in my mind about appending the JSON response, but for that when I am trying to apply forEach loop to my Json , no data is shown in the alert
Angular code
mainApp.controller('MultiCarouselController', function($scope, $http) {
$scope.products = [];
$http.get("/get_broad_category_products/?BroadCategory=BroadCategory3")
.success(function (response) {
$scope.products = response;
angular.forEach(products,function(value,key){
alert(key+'----'+value); // can see this alerts
});
}).error(function(){
console.log('Error happened ... ');
});
});
My JSON response is some junk data of Products
[
{
"sku": "d1cd71a8-9dc5-4724-b269-473ede28a1d7",
"selectedQtyOptions": [],
"selectedSize": "",
"description": "foolish interpolates trumpet monographs ferried inboards Forster tike grammatically sunroof vaporizing Sweden demure retouching completely robbing readies unloose guiltless tatty unobservant cuffs fortieth wither rigorously paradoxically snowmobiling charts clenching planning dealing lesions bicameral pertly chaffinches grumpiness private purled insanely attainment proposal Fatima execrates pshaws chars actuators turboprop soughed kicking majors conquistadores Cynthia septuagenarians kneecaps titans attractions larvas invigorating trunking Shevat recluse Trina slenderness kinking falsified logistically hogged skyrocketing ordinal avoiding trademarked underfoot garter sacrificial pricey nosedive bachelors deiced heave dictatorial muffing prayed rewinding recopied limpidly Crichton conversion chitterlings signets Aiken Froissart turnoff snowshoe forded spiralled underwriters flourishes Sade splicer transfusions cesspools lifelike ruckus showering paean voguish Buck copings Russell watchdog magneto pored height zodiac motherland backings Venus obeys scooters nonintervention dinosaur unashamedly anathema hibernate consumerism portended worked mystically existentialist dissatisfies badgers unanimously triplicated Jenny sagacity Windex snoopier nonplusing shovelling Assam putty darn Sulawesi Italians gunnery codify develops rhinos upwards Louise welled experiences socks pinky mewed Camille claimants swirl squattest ware parenthetic bonitoes hydrangeas decolonizing omit skyjacks Gorky financiers province flywheel southeastward Bayeux updated yowl Tulsidas macintosh sprees pralines systolic uncommoner cilium tromping Asimov heinous cordoned combated camerawomen syndrome identified prizefights heavyweight vertically reflector integrity Hebrides sepulchral loner parrot smooths candidness",
"selectedQty": "1",
"title": "viragoes",
"brand": "Brand0",
"images": [
{
"image0": "/media/products/f791a316ced7b3b774bd61e138197224.jpg"
}
],
"sizeQtyPrice": [
{
"discountAttributes": "chinos theosophy misdemeanor irrigates school Pullman sombrely suspect vortex baddest",
"measureUnit": "ltr",
"discountPercent": 4,
"mrp": 3102,
"qty": 7,
"size": 66
},
{
"discountAttributes": "Molotov absurd traces pounces contracts clarions thighbone Hesse parricide constrains",
"measureUnit": "m",
"discountPercent": 16,
"mrp": 2773,
"qty": 7,
"size": 18
},
{
"discountAttributes": "detainment gunnysack vied expropriation unobtrusive collectables embracing poster hexing governess",
"measureUnit": "m",
"discountPercent": 6,
"mrp": 9920,
"qty": 6,
"size": 69
}
],
"id": 9
},
{
"sku": "838660bb-7ab9-4f2a-8be7-9602a5801756",
"selectedQtyOptions": [],
"selectedSize": "",
"description": "agreeing vizier bleariest trig appliquéing copulating commissariats Balzac lunchtimes glittery quacking Leoncavallo heehawing Tampax lizards pegged nanosecond centigrade subplots tumbrils give jawed skits nickel discontinues impinged evangelized Platonist waterlines dams symposiums intercessor cognition heavier softener dromedaries bravos immobilize consciously Clemons patch klutzier Kirkpatrick caddying designs Dulles twelfths undemocratic isolationists infected ma homering soliciting minibus pluralism fraternity catalyzed Scorpio pandemonium waxwing starter infuses rebuttals spirals rerunning interrogatories Manuel whomsoever tenderized conjoint baronesses callower parenthetic plusses extend cockier Fokker dewlap Cowper Swammerdam secs hock relaxations Judas Canadian presidency lo wildness Philippe picture beekeeper lull manuals transnational yaw chloroformed perennials distinctive Nottingham antiquaries underneath parted nervously basemen observatories scrubbed encoder egalitarians winnow caddish Hawaiians brownstones robbing exhaustible antagonist benefactresses Plasticine Peace platypi Guzman stippled shuts peacemakers butterfly Bolton grout McCain Lebanon bounce oleander Balkans endearments snowfall spoonerisms furnaces inequities billowy jutting guffaw beautifully penis newtons snuffboxes j Angelita tinkles literature depicts insouciant scribblers blinker disobediently devotees primordial sixties Kalamazoo shear contest classes cripple edging exactest cheat invocation thrived drunkenness Fuller architectures sprite Lillian constricts tucking chastisements walruses guzzlers rejoinder apprenticeships pillory spendthrift omens spoonful contortions precociously intensely motorway guts cahoot sculptor paralytics reminisce meltdown trusts lady pronghorn scurried Campbell micron flawing foals nigher",
"selectedQty": "1",
"title": "smokier",
"brand": "Brand2",
"images": [
{
"image0": "/media/products/f51a649e72694d23962ee77a97872f0e.jpg"
}
],
"sizeQtyPrice": [
{
"discountAttributes": "Beerbohm earldom Stanley seconding hypertension Sayers miserly epitome retires ditching",
"measureUnit": "m",
"discountPercent": 15,
"mrp": 5065,
"qty": 6,
"size": 83
},
{
"discountAttributes": "confine Newman bagel cornflower rears generator goaded midweeks drain cigarillo",
"measureUnit": "Kg",
"discountPercent": 12,
"mrp": 2284,
"qty": 9,
"size": 13
},
{
"discountAttributes": "eerier fizzes lessened rotisserie developer Gray industrial callused convergences ampoule",
"measureUnit": "gms",
"discountPercent": 4,
"mrp": 6816,
"qty": 8,
"size": 18
}
],
"id": 14
}
]

products isn't defined. It should be $scope.products