MongoDB mongoimport gives error when importing large json file - json

I have a large JSON formatted file (254MB) which I am trying to import into MongoDB in Cloud9 IDE using mongoimport. Json file consists of mongodb documents each on its own row using newline character. The file looks like this:
...
{"t":"1358836264","p":"1.33470"}
{"t":"1358836265","p":"1.33475"}
{"t":"1358836271","p":"1.33477"}
{"t":"1358836272","p":"1.33481"}
{"t":"1358836274","p":"1.33475"}
{"t":"1358836275","p":"1.33478"}
{"t":"1358836288","p":"1.33480"}
{"t":"1358836291","p":"1.33481"}
{"t":"1358836294","p":"1.33481"}
{"t":"1358836295","p":"1.33478"}
...
I have tried:
mongoimport -d woption -c eurusd eurusddata.json
This gave me the following output with error:
connected to: 127.0.0.1
Sun Oct 5 16:51:15.094 Progress: 1636800/26345472 6%
Sun Oct 5 16:51:15.094 49600 16533/second
Sun Oct 5 16:51:18.088 Progress: 3375900/26345472 12%
Sun Oct 5 16:51:18.088 102300 17050/second
Sun Oct 5 16:51:21.089 Progress: 4867500/26345472 18%
Sun Oct 5 16:51:21.089 147500 16388/second
Sun Oct 5 16:51:24.103 Progress: 7728600/26345472 29%
Sun Oct 5 16:51:24.103 234200 19516/second
Sun Oct 5 16:51:27.093 Progress: 10467600/26345472 39%
Sun Oct 5 16:51:27.093 317200 21146/second
Sun Oct 5 16:51:30.094 Progress: 13312200/26345472 50%
Sun Oct 5 16:51:30.094 403400 22411/second
Sun Oct 5 16:51:33.302 Progress: 16038000/26345472 60%
Sun Oct 5 16:51:33.303 486000 23142/second
Sun Oct 5 16:51:36.088 Progress: 17341500/26345472 65%
Sun Oct 5 16:51:36.088 525500 21895/second
Sun Oct 5 16:51:39.004 Progress: 18526200/26345472 70%
Sun Oct 5 16:51:39.004 561400 20792/second
Sun Oct 5 16:51:42.032 Progress: 19067400/26345472 72%
Sun Oct 5 16:51:42.032 577800 19260/second
Sun Oct 5 16:51:45.088 Progress: 20829600/26345472 79%
Sun Oct 5 16:51:45.088 631200 19127/second
Sun Oct 5 16:51:48.071 Progress: 23007600/26345472 87%
Sun Oct 5 16:51:48.071 697200 19366/second
Sun Oct 5 16:51:51.914 Progress: 23443200/26345472 88%
Sun Oct 5 16:51:51.914 710400 18215/second
Sun Oct 5 16:51:54.103 Progress: 23611500/26345472 89%
Sun Oct 5 16:51:54.104 715500 17035/second
Sun Oct 5 16:51:57.709 Progress: 23967900/26345472 90%
Sun Oct 5 16:51:57.709 726300 16140/second
Sun Oct 5 16:52:00.096 Progress: 24538800/26345472 93%
Sun Oct 5 16:52:00.096 743600 15491/second
Sun Oct 5 16:52:03.088 Progress: 25548600/26345472 96%
Sun Oct 5 16:52:03.088 774200 15180/second
Sun Oct 5 16:52:04.644 exception:BSON representation of supplied JSON is too large: code FailedToParse: FailedToParse: Expecting ':': offset:21
Sun Oct 5 16:52:04.644
Sun Oct 5 16:52:04.644 check 9 798347
Sun Oct 5 16:52:05.061 imported 798347 objects
Sun Oct 5 16:52:05.061 ERROR: encountered 1 error(s)
Then I tried using --jsonArray at the end of previous command:
mongoimport -d woption -c eurusd eurusddata.json --jsonArray
This gave me the following output with error:
connected to: 127.0.0.1
Sun Oct 5 16:54:20.343 exception:JSONArray file too large
Sun Oct 5 16:54:20.444 warning: log line attempted (16384k) over max size(10k), printing beginning and end ... {"t":"1357070402","p":"1.32041"}
{"t":"1357070424","p":"1.32040"}
{"t":"1357070447","p":"1.32038"}
{"t":"1357070457","p":"1.32034"}
{"t":"1357070463","p":"1.32039"}
{"t":"1357070464","p":"1.32038"}
{"t":"1357070470","p":"1.32034"}
{"t":"1357070485","p":"1.32035"}
{"t":"1357070491","p":"1.32046"}
{"t":"1357070520","p":"1.32050"}
{"t":"1357070522","p":"1.32049"}
{"t":"1357070535","p":"1.32050"}
{"t":"1357070581","p":"1.32049"}
{"t":"1357070582","p":"1.32050"}
{"t":"1357070587","p":"1.32035"}
{"t":"1357070589","p":"1.32034"}
{"t":"1357070593","p":"1.32033"}
{"t":"1357070594","p":"1.32024"}
{"t":"1357070595","p":"1.32025"}
{"t":"1357070599","p":"1.32009"}
{"t":"1357070602","p":"1.32024"}
{"t":"1357070630","p":"1.32023"}
{"t":"1357070637","p":"1.32025"}
{"t":"1357070656","p":"1.32023"}
{"t":"1357070691","p":"1.32025"}
{"t":"1357070702","p":"1.32027"}
{"t":"1357070703","p":"1.32028"}
{"t":"1357070706","p":"1.32027"}
{"t":"1357070707","p":"1.32026"}
{"t":"1357070709","p":"1.32027"}
{"t":"1357070710","p":"1.32026"}
{"t":"1357070721","p":"1.32027"}
{"t":"1357070723","p":"1.32026"}
{"t":"1357070761","p":"1.32025"}
{"t":"1357070767","p":"1.32027"}
{"t":"1357070768","p":"1.32025"}
{"t":"1357070784","p":"1.32026"}
{"t":"1357070798","p":"1.32027"}
{"t":"1357070799","p":"1.32026"}
{"t":"1357070804","p":"1.32036"}
{"t":"1357070819","p":"1.32034"}
{"t":"1357070842","p":"1.32035"}
{"t":"1357070880","p":"1.32030"}
{"t":"1357070881","p":"1.32035"}
{"t":"1357070966","p":"1.32053"}
{"t":"1357070967","p":"1.32063"}
{"t":"1357070973","p":"1.32048"}
{"t":"1357070974","p":"1.32046"}
{"t":"1357070977","p":"1.32066"}
{"t":"1357070978","p":"1.32065"}
{"t":"1357070984","p":"1.32064"}
{"t":"1357070987","p":"1.32063"}
{"t":"1357070988","p":"1.32064"}
{"t":"1357070993","p":"1.32065"}
{"t":"1357070997","p":"1.32065"}
{"t":"1357070998","p":"1.32063"}
{"t":"1357071000","p":"1.32064"}
{"t":"1357071003","p":"1.32064"}
{"t":"1357071011","p":"1.32065"}
{"t":"1357071012","p":"1.32069"}
{"t":"1357071013","p":"1.32069"}
{"t":"1357071023","p":"1.32068"}
{"t":"1357071024","p":"1.32068"}
{"t":"1357071026","p":"1.32072"}
{"t":"1357071027","p":"1.32070"}
{"t":"1357071028","p":"1.32069"}
{"t":"1357071047","p":"1.32068"}
{"t":"1357071048","p":"1.32069"}
{"t":"1357071070","p":"1.32072"}
{"t":"1357071071","p":"1.32069"}
{"t":"1357071077","p":"1.32068"}
{"t":"1357071078","p":"1.32069"}
{"t":"1357071085","p":"1.32068"}
{"t":"1357071086","p":"1.32069"}
{"t":"1357071095","p":"1.32072"}
{"t":"1357071096","p":"1.32069"}
{"t":"1357071097","p":"1.32070"}
{"t":"1357071104","p":"1.32069"}
{"t":"1357071105","p":"1.32069"}
{"t":"1357071114","p":"1.32068"}
{"t":"1357071125","p":"1.32068"}
{"t":"1357071132","p":"1.32072"}
{"t":"1357071133","p":"1.32068"}
{"t":"1357071134","p":"1.32069"}
{"t":"1357071136","p":"1.32043"}
{"t":"1357071137","p":"1.32049"}
{"t":"1357071139","p":"1.32052"}
{"t":"1357071140","p":"1.32058"}
{"t":"1357071156","p":"1.32052"}
{"t":"1357071163","p":"1.32050"}
{"t":"1357071174","p":"1.32052"}
{"t":"1357071176","p":"1.32055"}
{"t":"1357071178","p":"1.32053"}
{"t":"1357071185","p":"1.32054"}
{"t":"1357071187","p":"1.32053"}
{"t":"1357071189","p":"1.32025"}
{"t":"1357071190","p":"1.32023"}
{"t":"1357071191","p":"1.32045"}
{"t":"1357071192","p":"1.32044"}
{"t":"1357071193","p":"1.32045"}
{"t":"1357071197","p":"1.32044"}
{"t":"1357071198","p":"1.32043"}
{"t":"1357071199","p":"1.32004"}
{"t":"13570712 .......... ":"1358836127","p":"1.33455"}
{"t":"1358836128","p":"1.33454"}
{"t":"1358836129","p":"1.33455"}
{"t":"1358836130","p":"1.33458"}
{"t":"1358836131","p":"1.33463"}
{"t":"1358836132","p":"1.33462"}
{"t":"1358836133","p":"1.33460"}
{"t":"1358836134","p":"1.33459"}
{"t":"1358836135","p":"1.33464"}
{"t":"1358836137","p":"1.33463"}
{"t":"1358836139","p":"1.33461"}
{"t":"1358836140","p":"1.33462"}
{"t":"1358836142","p":"1.33463"}
{"t":"1358836143","p":"1.33459"}
{"t":"1358836144","p":"1.33461"}
{"t":"1358836146","p":"1.33462"}
{"t":"1358836147","p":"1.33466"}
{"t":"1358836148","p":"1.33473"}
{"t":"1358836149","p":"1.33475"}
{"t":"1358836151","p":"1.33479"}
{"t":"1358836152","p":"1.33484"}
{"t":"1358836153","p":"1.33492"}
{"t":"1358836154","p":"1.33488"}
{"t":"1358836155","p":"1.33492"}
{"t":"1358836156","p":"1.33489"}
{"t":"1358836157","p":"1.33482"}
{"t":"1358836158","p":"1.33483"}
{"t":"1358836159","p":"1.33480"}
{"t":"1358836160","p":"1.33479"}
{"t":"1358836161","p":"1.33482"}
{"t":"1358836162","p":"1.33481"}
{"t":"1358836163","p":"1.33481"}
{"t":"1358836166","p":"1.33480"}
{"t":"1358836167","p":"1.33480"}
{"t":"1358836168","p":"1.33475"}
{"t":"1358836169","p":"1.33454"}
{"t":"1358836170","p":"1.33454"}
{"t":"1358836172","p":"1.33459"}
{"t":"1358836173","p":"1.33456"}
{"t":"1358836174","p":"1.33455"}
{"t":"1358836175","p":"1.33457"}
{"t":"1358836176","p":"1.33459"}
{"t":"1358836177","p":"1.33460"}
{"t":"1358836178","p":"1.33462"}
{"t":"1358836179","p":"1.33458"}
{"t":"1358836180","p":"1.33459"}
{"t":"1358836181","p":"1.33456"}
{"t":"1358836182","p":"1.33458"}
{"t":"1358836183","p":"1.33457"}
{"t":"1358836184","p":"1.33459"}
{"t":"1358836185","p":"1.33460"}
{"t":"1358836186","p":"1.33467"}
{"t":"1358836189","p":"1.33466"}
{"t":"1358836190","p":"1.33472"}
{"t":"1358836191","p":"1.33470"}
{"t":"1358836192","p":"1.33470"}
{"t":"1358836193","p":"1.33469"}
{"t":"1358836194","p":"1.33465"}
{"t":"1358836195","p":"1.33466"}
{"t":"1358836196","p":"1.33461"}
{"t":"1358836197","p":"1.33460"}
{"t":"1358836198","p":"1.33466"}
{"t":"1358836199","p":"1.33465"}
{"t":"1358836200","p":"1.33466"}
{"t":"1358836201","p":"1.33458"}
{"t":"1358836202","p":"1.33457"}
{"t":"1358836203","p":"1.33453"}
{"t":"1358836204","p":"1.33454"}
{"t":"1358836205","p":"1.33456"}
{"t":"1358836210","p":"1.33455"}
{"t":"1358836211","p":"1.33450"}
{"t":"1358836214","p":"1.33449"}
{"t":"1358836216","p":"1.33451"}
{"t":"1358836217","p":"1.33452"}
{"t":"1358836218","p":"1.33457"}
{"t":"1358836220","p":"1.33458"}
{"t":"1358836221","p":"1.33457"}
{"t":"1358836222","p":"1.33456"}
{"t":"1358836223","p":"1.33457"}
{"t":"1358836226","p":"1.33460"}
{"t":"1358836229","p":"1.33459"}
{"t":"1358836230","p":"1.33458"}
{"t":"1358836236","p":"1.33461"}
{"t":"1358836237","p":"1.33466"}
{"t":"1358836240","p":"1.33467"}
{"t":"1358836241","p":"1.33467"}
{"t":"1358836242","p":"1.33463"}
{"t":"1358836243","p":"1.33466"}
{"t":"1358836244","p":"1.33467"}
{"t":"1358836245","p":"1.33468"}
{"t":"1358836250","p":"1.33468"}
{"t":"1358836256","p":"1.33469"}
{"t":"1358836263","p":"1.33472"}
{"t":"1358836264","p":"1.33470"}
{"t":"1358836265","p":"1.33475"}
{"t":"1358836271","p":"1.33477"}
{"t":"1358836272","p":"1.33481"}
{"t":"1358836274","p":"1.33475"}
{"t":"1358836275","p":"1.33478"}
{"t":"1358836288","p":"1.33480"}
{"t":"1358836291","p":"1.33481"}
{"t":"1358836294","p":"1.33481"}
{"t":"1358836295","p":"1.33478"}
{"t":"1358836296
Sun Oct 5 16:54:20.445 check 0 0
Sun Oct 5 16:54:20.445 imported 0 objects
Sun Oct 5 16:54:20.445 ERROR: encountered 1 error(s)
I then tried using something like this:
mongoimport -d woption -c eurusd < /home/ubuntu/workspace/eurusddata.json
This gave me the following output with error:
connected to: 127.0.0.1
Sun Oct 5 16:58:08.107 77400 25800/second
Sun Oct 5 16:58:11.096 136300 22716/second
Sun Oct 5 16:58:14.107 177700 19744/second
Sun Oct 5 16:58:17.093 232900 19408/second
Sun Oct 5 16:58:20.125 280400 18693/second
Sun Oct 5 16:58:23.103 343500 19083/second
Sun Oct 5 16:58:26.091 424800 20228/second
Sun Oct 5 16:58:29.095 482800 20116/second
Sun Oct 5 16:58:32.097 517900 19181/second
Sun Oct 5 16:58:35.088 566700 18890/second
Sun Oct 5 16:58:38.091 608700 18445/second
Sun Oct 5 16:58:41.098 664700 18463/second
Sun Oct 5 16:58:44.088 718300 18417/second
Sun Oct 5 16:58:47.089 776800 18495/second
Sun Oct 5 16:58:47.848 exception:BSON representation of supplied JSON is too large: code FailedToParse: FailedToParse: Expecting ':': offset:21
Sun Oct 5 16:58:47.848
Sun Oct 5 16:58:47.848 check 9 798347
Sun Oct 5 16:58:48.852 imported 798347 objects
Sun Oct 5 16:58:48.852 ERROR: encountered 1 error(s)
Also tried using previous command with --jsonArray:
mongoimport -d woption -c eurusd < /home/ubuntu/workspace/eurusddata.json --jsonArray
This gave me the following output with error:
connected to: 127.0.0.1
Sun Oct 5 16:59:35.063 exception:JSONArray file too large
Sun Oct 5 16:59:35.101 warning: log line attempted (16384k) over max size(10k), printing beginning and end ... {"t":"1357070402","p":"1.32041"}
{"t":"1357070424","p":"1.32040"}
{"t":"1357070447","p":"1.32038"}
{"t":"1357070457","p":"1.32034"}
{"t":"1357070463","p":"1.32039"}
{"t":"1357070464","p":"1.32038"}
{"t":"1357070470","p":"1.32034"}
{"t":"1357070485","p":"1.32035"}
{"t":"1357070491","p":"1.32046"}
{"t":"1357070520","p":"1.32050"}
{"t":"1357070522","p":"1.32049"}
{"t":"1357070535","p":"1.32050"}
{"t":"1357070581","p":"1.32049"}
{"t":"1357070582","p":"1.32050"}
{"t":"1357070587","p":"1.32035"}
{"t":"1357070589","p":"1.32034"}
{"t":"1357070593","p":"1.32033"}
{"t":"1357070594","p":"1.32024"}
{"t":"1357070595","p":"1.32025"}
{"t":"1357070599","p":"1.32009"}
{"t":"1357070602","p":"1.32024"}
{"t":"1357070630","p":"1.32023"}
{"t":"1357070637","p":"1.32025"}
{"t":"1357070656","p":"1.32023"}
{"t":"1357070691","p":"1.32025"}
{"t":"1357070702","p":"1.32027"}
{"t":"1357070703","p":"1.32028"}
{"t":"1357070706","p":"1.32027"}
{"t":"1357070707","p":"1.32026"}
{"t":"1357070709","p":"1.32027"}
{"t":"1357070710","p":"1.32026"}
{"t":"1357070721","p":"1.32027"}
{"t":"1357070723","p":"1.32026"}
{"t":"1357070761","p":"1.32025"}
{"t":"1357070767","p":"1.32027"}
{"t":"1357070768","p":"1.32025"}
{"t":"1357070784","p":"1.32026"}
{"t":"1357070798","p":"1.32027"}
{"t":"1357070799","p":"1.32026"}
{"t":"1357070804","p":"1.32036"}
{"t":"1357070819","p":"1.32034"}
{"t":"1357070842","p":"1.32035"}
{"t":"1357070880","p":"1.32030"}
{"t":"1357070881","p":"1.32035"}
{"t":"1357070966","p":"1.32053"}
{"t":"1357070967","p":"1.32063"}
{"t":"1357070973","p":"1.32048"}
{"t":"1357070974","p":"1.32046"}
{"t":"1357070977","p":"1.32066"}
{"t":"1357070978","p":"1.32065"}
{"t":"1357070984","p":"1.32064"}
{"t":"1357070987","p":"1.32063"}
{"t":"1357070988","p":"1.32064"}
{"t":"1357070993","p":"1.32065"}
{"t":"1357070997","p":"1.32065"}
{"t":"1357070998","p":"1.32063"}
{"t":"1357071000","p":"1.32064"}
{"t":"1357071003","p":"1.32064"}
{"t":"1357071011","p":"1.32065"}
{"t":"1357071012","p":"1.32069"}
{"t":"1357071013","p":"1.32069"}
{"t":"1357071023","p":"1.32068"}
{"t":"1357071024","p":"1.32068"}
{"t":"1357071026","p":"1.32072"}
{"t":"1357071027","p":"1.32070"}
{"t":"1357071028","p":"1.32069"}
{"t":"1357071047","p":"1.32068"}
{"t":"1357071048","p":"1.32069"}
{"t":"1357071070","p":"1.32072"}
{"t":"1357071071","p":"1.32069"}
{"t":"1357071077","p":"1.32068"}
{"t":"1357071078","p":"1.32069"}
{"t":"1357071085","p":"1.32068"}
{"t":"1357071086","p":"1.32069"}
{"t":"1357071095","p":"1.32072"}
{"t":"1357071096","p":"1.32069"}
{"t":"1357071097","p":"1.32070"}
{"t":"1357071104","p":"1.32069"}
{"t":"1357071105","p":"1.32069"}
{"t":"1357071114","p":"1.32068"}
{"t":"1357071125","p":"1.32068"}
{"t":"1357071132","p":"1.32072"}
{"t":"1357071133","p":"1.32068"}
{"t":"1357071134","p":"1.32069"}
{"t":"1357071136","p":"1.32043"}
{"t":"1357071137","p":"1.32049"}
{"t":"1357071139","p":"1.32052"}
{"t":"1357071140","p":"1.32058"}
{"t":"1357071156","p":"1.32052"}
{"t":"1357071163","p":"1.32050"}
{"t":"1357071174","p":"1.32052"}
{"t":"1357071176","p":"1.32055"}
{"t":"1357071178","p":"1.32053"}
{"t":"1357071185","p":"1.32054"}
{"t":"1357071187","p":"1.32053"}
{"t":"1357071189","p":"1.32025"}
{"t":"1357071190","p":"1.32023"}
{"t":"1357071191","p":"1.32045"}
{"t":"1357071192","p":"1.32044"}
{"t":"1357071193","p":"1.32045"}
{"t":"1357071197","p":"1.32044"}
{"t":"1357071198","p":"1.32043"}
{"t":"1357071199","p":"1.32004"}
{"t":"13570712 .......... ":"1358836127","p":"1.33455"}
{"t":"1358836128","p":"1.33454"}
{"t":"1358836129","p":"1.33455"}
{"t":"1358836130","p":"1.33458"}
{"t":"1358836131","p":"1.33463"}
{"t":"1358836132","p":"1.33462"}
{"t":"1358836133","p":"1.33460"}
{"t":"1358836134","p":"1.33459"}
{"t":"1358836135","p":"1.33464"}
{"t":"1358836137","p":"1.33463"}
{"t":"1358836139","p":"1.33461"}
{"t":"1358836140","p":"1.33462"}
{"t":"1358836142","p":"1.33463"}
{"t":"1358836143","p":"1.33459"}
{"t":"1358836144","p":"1.33461"}
{"t":"1358836146","p":"1.33462"}
{"t":"1358836147","p":"1.33466"}
{"t":"1358836148","p":"1.33473"}
{"t":"1358836149","p":"1.33475"}
{"t":"1358836151","p":"1.33479"}
{"t":"1358836152","p":"1.33484"}
{"t":"1358836153","p":"1.33492"}
{"t":"1358836154","p":"1.33488"}
{"t":"1358836155","p":"1.33492"}
{"t":"1358836156","p":"1.33489"}
{"t":"1358836157","p":"1.33482"}
{"t":"1358836158","p":"1.33483"}
{"t":"1358836159","p":"1.33480"}
{"t":"1358836160","p":"1.33479"}
{"t":"1358836161","p":"1.33482"}
{"t":"1358836162","p":"1.33481"}
{"t":"1358836163","p":"1.33481"}
{"t":"1358836166","p":"1.33480"}
{"t":"1358836167","p":"1.33480"}
{"t":"1358836168","p":"1.33475"}
{"t":"1358836169","p":"1.33454"}
{"t":"1358836170","p":"1.33454"}
{"t":"1358836172","p":"1.33459"}
{"t":"1358836173","p":"1.33456"}
{"t":"1358836174","p":"1.33455"}
{"t":"1358836175","p":"1.33457"}
{"t":"1358836176","p":"1.33459"}
{"t":"1358836177","p":"1.33460"}
{"t":"1358836178","p":"1.33462"}
{"t":"1358836179","p":"1.33458"}
{"t":"1358836180","p":"1.33459"}
{"t":"1358836181","p":"1.33456"}
{"t":"1358836182","p":"1.33458"}
{"t":"1358836183","p":"1.33457"}
{"t":"1358836184","p":"1.33459"}
{"t":"1358836185","p":"1.33460"}
{"t":"1358836186","p":"1.33467"}
{"t":"1358836189","p":"1.33466"}
{"t":"1358836190","p":"1.33472"}
{"t":"1358836191","p":"1.33470"}
{"t":"1358836192","p":"1.33470"}
{"t":"1358836193","p":"1.33469"}
{"t":"1358836194","p":"1.33465"}
{"t":"1358836195","p":"1.33466"}
{"t":"1358836196","p":"1.33461"}
{"t":"1358836197","p":"1.33460"}
{"t":"1358836198","p":"1.33466"}
{"t":"1358836199","p":"1.33465"}
{"t":"1358836200","p":"1.33466"}
{"t":"1358836201","p":"1.33458"}
{"t":"1358836202","p":"1.33457"}
{"t":"1358836203","p":"1.33453"}
{"t":"1358836204","p":"1.33454"}
{"t":"1358836205","p":"1.33456"}
{"t":"1358836210","p":"1.33455"}
{"t":"1358836211","p":"1.33450"}
{"t":"1358836214","p":"1.33449"}
{"t":"1358836216","p":"1.33451"}
{"t":"1358836217","p":"1.33452"}
{"t":"1358836218","p":"1.33457"}
{"t":"1358836220","p":"1.33458"}
{"t":"1358836221","p":"1.33457"}
{"t":"1358836222","p":"1.33456"}
{"t":"1358836223","p":"1.33457"}
{"t":"1358836226","p":"1.33460"}
{"t":"1358836229","p":"1.33459"}
{"t":"1358836230","p":"1.33458"}
{"t":"1358836236","p":"1.33461"}
{"t":"1358836237","p":"1.33466"}
{"t":"1358836240","p":"1.33467"}
{"t":"1358836241","p":"1.33467"}
{"t":"1358836242","p":"1.33463"}
{"t":"1358836243","p":"1.33466"}
{"t":"1358836244","p":"1.33467"}
{"t":"1358836245","p":"1.33468"}
{"t":"1358836250","p":"1.33468"}
{"t":"1358836256","p":"1.33469"}
{"t":"1358836263","p":"1.33472"}
{"t":"1358836264","p":"1.33470"}
{"t":"1358836265","p":"1.33475"}
{"t":"1358836271","p":"1.33477"}
{"t":"1358836272","p":"1.33481"}
{"t":"1358836274","p":"1.33475"}
{"t":"1358836275","p":"1.33478"}
{"t":"1358836288","p":"1.33480"}
{"t":"1358836291","p":"1.33481"}
{"t":"1358836294","p":"1.33481"}
{"t":"1358836295","p":"1.33478"}
{"t":"1358836296
Sun Oct 5 16:59:35.101 check 0 0
Sun Oct 5 16:59:35.102 imported 0 objects
Sun Oct 5 16:59:35.102 ERROR: encountered 1 error(s)
What am I doing wrong or what am I missing here? How can I import large json files to MongoDB?


According to the message:
Sun Oct 5 16:52:03.088 Progress: 25548600/26345472 96%
Sun Oct 5 16:52:03.088 774200 15180/second
Sun Oct 5 16:52:04.644 exception:BSON representation of supplied JSON is too large: code FailedToParse: FailedToParse: Expecting ':': offset:21
I think there are some mistakes near the end of your imported file.
For {"t":"1358836291","p":"1.33481"}, the offset of second : is 21. So, please check syntax of documents near the end (96% ~ 100%) of imported file.
By the way, --jsonArray is improper to use here because it has a limit of 16MB.


Change your command from mongoimport -d woption -c eurusd eurusddata.json --jsonArray to mongoimport -d woption -c eurusd --file eurusddata.json --jsonArray and this will work. Hope this helps. Update if this doesn't work.


Try this option: --batchSize 1
Like:
mongoimport -d woption -c eurusd eurusddata.json --batchSize 1

Related

Unable to start mysql on EC2 instance, I keep receiving apparmor error

I am unable to start mysql on my EC2 instance. Receiving following errors
Oct 28 05:21:08 ip-172-30-1-66 kernel: [ 841.115678] init: mysql main process (19029) terminated with status 7
Oct 28 05:21:08 ip-172-30-1-66 kernel: [ 841.115687] init: mysql main process ended, respawning
Oct 28 05:21:09 ip-172-30-1-66 kernel: [ 842.011098] init: mysql post-start process (19030) terminated with status 1
Oct 28 05:21:09 ip-172-30-1-66 kernel: [ 842.016704] type=1400 audit(1540704069.482:278): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/sbin/mysqld" pid=19084 comm="apparmor_parser"
This is on an Ubuntu system
# uname -a
Linux ip-172-30-1-66 3.13.0-91-generic #138-Ubuntu SMP Fri Jun 24 17:00:34 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Server-sent event received after 3-4 seconds

While reading about Server-Sent Events from this page, I got confused about the timing of events. Basically, the example shown has a PHP script sending the system time to the web page:
<?php
header('Content-Type: text/event-stream');
header('Cache-Control: no-cache');
$time = date('r');
echo "data: The server time is: {$time}\n\n";
flush();
?>
While the page receives and renders it:
if(typeof(EventSource) !== "undefined") {
var source = new EventSource("demo_sse.php");
source.onmessage = function(event) {
document.getElementById("result").innerHTML += event.data + "<br>";
};
} else {
document.getElementById("result").innerHTML = "Sorry, your browser does not support server-sent events...";
}
My confusion is that the server seems to be sending messages every 3-4 seconds:
The server time is: Tue, 20 Sep 2016 11:55:12 -0400
The server time is: Tue, 20 Sep 2016 11:55:16 -0400
The server time is: Tue, 20 Sep 2016 11:55:20 -0400
The server time is: Tue, 20 Sep 2016 11:55:23 -0400
The server time is: Tue, 20 Sep 2016 11:55:28 -0400
The server time is: Tue, 20 Sep 2016 11:55:32 -0400
The server time is: Tue, 20 Sep 2016 11:55:35 -0400
The server time is: Tue, 20 Sep 2016 11:55:39 -0400
The server time is: Tue, 20 Sep 2016 11:55:43 -0400
The server time is: Tue, 20 Sep 2016 11:55:46 -0400
The server time is: Tue, 20 Sep 2016 11:55:50 -0400
The server time is: Tue, 20 Sep 2016 11:55:53 -0400
The server time is: Tue, 20 Sep 2016 11:55:57 -0400
The server time is: Tue, 20 Sep 2016 11:56:01 -0400
The server time is: Tue, 20 Sep 2016 11:56:04 -0400
The server time is: Tue, 20 Sep 2016 11:56:08 -0400
The server time is: Tue, 20 Sep 2016 11:56:12 -0400
The server time is: Tue, 20 Sep 2016 11:56:15 -0400
However, I don't see this delay either at the server end or at the client end. Is it the network lag between the website's server and my browser? Or is it something else?

Default retry is 3 seconds as shown on http://www.html5rocks.com/en/tutorials/eventsource/basics/
Check out the "Controlling the Reconnection-timeout section" on that link.
You may customize it by sending a line retry:100
to enforce just 100ms retry time interval.

Error occurred while closing statement in dbal

In my silex web aplication I have this function:
function obtener_visible($conn, $articulo, $wid)
{
$q= $conn
->prepare('call vn2008.article_visible_single(?, ?)');
$q->execute(array(0 => $wid, 1=>$articulo));
$result = $q->fetch();
$conn->close();
return $result['suma'];
}
But when I call this function, I receive this error:
[Wed Oct 1 07:47:01 2014] PHP Warning: obtener_visible(): Error occurred while closing statement in /home/nelo/web/silex/almacen.php on line 0
[Wed Oct 1 07:47:01 2014] PHP Stack trace:
[Wed Oct 1 07:47:01 2014] PHP 1. {main}() /home/nelo/web/silex/index.php:0
[Wed Oct 1 07:47:01 2014] PHP 2. Silex\Application->run() /home/nelo/web/silex/index.php:38
[Wed Oct 1 07:47:01 2014] PHP 3. Silex\Application->handle() /home/nelo/web/silex/vendor/silex/silex/src/Silex/Application.php:481
[Wed Oct 1 07:47:01 2014] PHP 4. Symfony\Component\HttpKernel\HttpKernel->handle() /home/nelo/web/silex/vendor/silex/silex/src/Silex/Application.php:504
[Wed Oct 1 07:47:01 2014] PHP 5. Symfony\Component\HttpKernel\HttpKernel->handleRaw() /home/nelo/web/silex/vendor/symfony/http-kernel/Symfony/Component/HttpKernel/HttpKernel.php:61
[Wed Oct 1 07:47:01 2014] PHP 6. call_user_func_array() /home/nelo/web/silex/vendor/symfony/http-kernel/Symfony/Component/HttpKernel/HttpKernel.php:117
[Wed Oct 1 07:47:01 2014] PHP 7. {closure:/home/nelo/web/silex/index.php:18-22}() /home/nelo/web/silex/vendor/symfony/http-kernel/Symfony/Component/HttpKernel/HttpKernel.php:117
[Wed Oct 1 07:47:01 2014] PHP 8. executeFacade() /home/nelo/web/silex/index.php:20
[Wed Oct 1 07:47:01 2014] PHP 9. Verdnatura\Almacen\obtener_articulo->execute() /home/nelo/web/silex/facade.php:28
[Wed Oct 1 07:47:01 2014] PHP 10. obtener_visible() /home/nelo/web/silex/almacen/obtener_articulo.php:56
Can someone help me solve this error?

Vagrant Chef Recipe Checksum Calculations

I am running a mysql recipe that is failing. When I do vagrant up after a halt it claims that grants.sql template's checksum has changed causing it to re-run when it shouldn't.
[default] [Wed, 28 Mar 2012 12:58:48 -0700] INFO: Processing template[/etc/mysql/grants.sql] action create (mysql::server line 128)
: stdout
[default] [Wed, 28 Mar 2012 12:58:48 -0700] DEBUG: Current content's checksum: 3992e44304b56cebdbd4bf23183ddd78f877539c025227546e19098b0b5872ca
: stdout
[default] [Wed, 28 Mar 2012 12:58:48 -0700] DEBUG: Rendered content's checksum: f967f212b3e7b25a08ed35d086938846c188f6e9980a1ecc42635136841587a4
: stdout
[default] [Wed, 28 Mar 2012 12:58:48 -0700] INFO: template[/etc/mysql/grants.sql] backed up to /var/chef/backup/etc/mysql/grants.sql.chef-20120328125848
: stdout
[default] [Wed, 28 Mar 2012 12:58:48 -0700] INFO: template[/etc/mysql/grants.sql] updated content
: stdout
[default] [Wed, 28 Mar 2012 12:58:48 -0700] INFO: template[/etc/mysql/grants.sql] sending run action to execute[mysql-install-privileges] (immediate)
: stdout
[default] [Wed, 28 Mar 2012 12:58:48 -0700] INFO: Processing execute[mysql-install-privileges] action run (mysql::server line 137)
: stdout
[default] [Wed, 28 Mar 2012 12:58:48 -0700] INFO: execute[mysql-install-privileges] sh(/usr/bin/mysql -u root -p"evanta" < /etc/mysql/grants.sql)
: stdout
[default] [Wed, 28 Mar 2012 12:58:48 -0700] ERROR: execute[mysql-install-privileges] (mysql::server line 137) has had an error
[Wed, 28 Mar 2012 12:58:48 -0700] ERROR: template[/etc/mysql/grants.sql] (/tmp/vagrant-chef-1/chef-solo-1/mysql/recipes/server.rb:128:in `rescue in from_file') had an error:
execute[mysql-install-privileges] (mysql::server line 137) had an error: Chef::Exceptions::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of /usr/bin/mysql -u root -p"evanta" < /etc/mysql/grants.sql ----
STDOUT:
STDERR: ERROR 1396 (HY000) at line 12: Operation CREATE USER failed for 'root'#'%'
---- End output of /usr/bin/mysql -u root -p"evanta" < /etc/mysql/grants.sql ----
Ran /usr/bin/mysql -u root -p"evanta" < /etc/mysql/grants.sql returned 1
Any ideas how these checksums are completed and how to fix this?

The best way to debug these issues is to take a look at the new file it created, in this case at /etc/mysql/grants.sql, and then to look at the backup at /var/chef/backup/etc/mysql/grants.sql.chef-20120328125848 (from the logs you posted). The backup is always made, so you can compare the contents of the two, and proceed to fix the Chef recipe to make sure it generates the same content.

YAJL stream parsing and Twitter streaming API

Twitter streaming API returns blocks of JSON, but my YAJL parser stops after the first one. I guess this is because every block of JSON is independant (i.e: not in a global array), so YAJL has no way of knowing it's not done.
How can I handle this ? Here is the sample of what I get through the stream :
{"user":{"statuses_count":357,"profile_link_color":"93A644","profile_sidebar_border_color":"eeeeee","followers_count":28,"contributors_enabled":false,"profile_use_background_image":true,"created_at":"Fri Jun 04 22:45:23 +0000 2010","location":"Brazil","verified":false,"profile_background_color":"B2DFDA","follow_request_sent":null,"profile_background_image_url":"http:\/\/a3.twimg.com\/profile_background_images\/201416971\/egito.jpg","description":"Dare to defy the status quo!!! For a free Egypt!","is_translator":false,"favourites_count":14,"time_zone":"Brasilia","profile_text_color":"333333","protected":false,"url":null,"profile_image_url":"http:\/\/a2.twimg.com\/profile_images\/1233185161\/Flag-Pins-Brazil-Egypt_normal.jpg","notifications":null,"profile_sidebar_fill_color":"ffffff","name":"lu costa","id_str":"152038309","listed_count":0,"lang":"en","profile_background_tile":true,"screen_name":"dacostalu","id":152038309,"show_all_inline_media":false,"following":null,"geo_enabled":false,"utc_offset":-10800,"friends_count":35},"coordinates":null,"retweet_count":49,"truncated":false,"text":"RT #arwasm: Protesters are calling for volunteers to be at the entrances to #tahrir. Entrances are more exposed than yesterday. #jan25 # ...","favorited":false,"created_at":"Sat Feb 05 15:13:00 +0000 2011","in_reply_to_user_id":null,"in_reply_to_status_id":null,"source":"web","in_reply_to_screen_name":null,"in_reply_to_status_id_str":null,"entities":{"hashtags":[{"indices":[76,83],"text":"tahrir"},{"indices":[128,134],"text":"jan25"}],"user_mentions":[{"indices":[3,10],"name":"Arwa Mahmoud","screen_name":"arwasm","id_str":"63693027","id":63693027}],"urls":[]},"contributors":null,"place":null,"geo":null,"retweeted":false,"in_reply_to_user_id_str":null,"id_str":"33905938754834432","retweeted_status":{"user":{"statuses_count":3057,"profile_link_color":"990000","profile_sidebar_border_color":"DFDFDF","followers_count":1468,"contributors_enabled":false,"profile_use_background_image":true,"created_at":"Fri Aug 07 11:21:27 +0000 2009","location":"Cairo, Egypt","verified":false,"profile_background_color":"EBEBEB","follow_request_sent":null,"profile_background_image_url":"http:\/\/a2.twimg.com\/profile_background_images\/70305782\/even_smaller.jpg","description":"Writer, blogger, mountaineer wannabe. Author of A Battle of Creed & \u0642\u062a\u0627\u0644 \u062d\u0632\u0628 \u0627\u0644\u0644\u0647. Falls head over heals with every mountain she meets!","is_translator":false,"favourites_count":0,"time_zone":"Santiago","profile_text_color":"333333","protected":false,"url":"http:\/\/www.alhurr.com","profile_image_url":"http:\/\/a0.twimg.com\/profile_images\/1233059668\/e296709a-cb86-45fd-aa75-89d3fa8a02ac_normal.png","notifications":null,"profile_sidebar_fill_color":"F3F3F3","name":"Arwa Mahmoud","id_str":"63693027","listed_count":156,"lang":"en","profile_background_tile":true,"screen_name":"arwasm","id":63693027,"show_all_inline_media":false,"following":null,"geo_enabled":false,"utc_offset":-14400,"friends_count":182},"coordinates":null,"retweet_count":49,"truncated":false,"text":"Protesters are calling for volunteers to be at the entrances to #tahrir. Entrances are more exposed than yesterday. #jan25 #egypt","favorited":false,"created_at":"Sat Feb 05 14:40:39 +0000 2011","in_reply_to_user_id":null,"in_reply_to_status_id":null,"source":"\u003Ca href=\"http:\/\/blackberry.com\/twitter\" rel=\"nofollow\"\u003ETwitter for BlackBerry\u00ae\u003C\/a\u003E","in_reply_to_screen_name":null,"in_reply_to_status_id_str":null,"entities":{"hashtags":[{"indices":[64,71],"text":"tahrir"},{"indices":[116,122],"text":"jan25"},{"indices":[123,129],"text":"egypt"}],"user_mentions":[],"urls":[]},"contributors":null,"place":null,"geo":null,"retweeted":false,"in_reply_to_user_id_str":null,"id_str":"33897797342199808","id":33897797342199808},"id":33905938754834432}
{"user":{"follow_request_sent":null,"profile_link_color":"0099B9","is_translator":false,"profile_sidebar_border_color":"5ED4DC","followers_count":528,"profile_use_background_image":true,"created_at":"Sun Mar 08 04:52:10 +0000 2009","location":"SE USA","listed_count":7,"profile_background_color":"0099B9","show_all_inline_media":false,"geo_enabled":false,"profile_background_image_url":"http:\/\/a0.twimg.com\/profile_background_images\/17691490\/Bloom.jpg","friends_count":550,"description":"tech, motorcycles, photography, music, green, generally curious","statuses_count":6008,"favourites_count":95,"time_zone":"Central Time (US & Canada)","profile_text_color":"3C3940","protected":false,"url":null,"profile_image_url":"http:\/\/a3.twimg.com\/profile_images\/287875274\/Speedo_normal.jpg","notifications":null,"profile_sidebar_fill_color":"95E8EC","name":"Henk Sijgers","id_str":"23277821","contributors_enabled":false,"lang":"en","profile_background_tile":false,"screen_name":"henksijgers","id":23277821,"following":null,"verified":false,"utc_offset":-21600},"coordinates":null,"retweet_count":6,"truncated":true,"text":"RT #SultanAlQassemi: LA Times: Al Jazeera English hopes coverage of Egypt will boost reach in US http:\/\/lat.ms\/gmb15c I salute #AJEnglis ...","favorited":false,"created_at":"Sat Feb 05 15:13:00 +0000 2011","in_reply_to_user_id":null,"in_reply_to_status_id":null,"source":"\u003Ca href=\"http:\/\/itunes.apple.com\/app\/twitter\/id333903271?mt=8\" rel=\"nofollow\"\u003ETwitter for iPad\u003C\/a\u003E","in_reply_to_screen_name":null,"in_reply_to_status_id_str":null,"entities":{"hashtags":[],"urls":[{"indices":[97,117],"url":"http:\/\/lat.ms\/gmb15c","expanded_url":null}],"user_mentions":[{"indices":[3,19],"name":"Sultan Al Qassemi","screen_name":"SultanAlQassemi","id_str":"46744791","id":46744791},{"indices":[127,136],"name":"Mahandini Teukie","screen_name":"ajenglis","id_str":"241013489","id":241013489}]},"contributors":null,"place":null,"geo":null,"retweeted":false,"in_reply_to_user_id_str":null,"id_str":"33905938419159040","retweeted_status":{"user":{"follow_request_sent":null,"profile_link_color":"009999","is_translator":false,"profile_sidebar_border_color":"eeeeee","followers_count":27255,"profile_use_background_image":true,"created_at":"Fri Jun 12 20:47:01 +0000 2009","location":"\u00dcT: 25.206282,55.281949","listed_count":1794,"profile_background_color":"131516","show_all_inline_media":true,"geo_enabled":false,"profile_background_image_url":"http:\/\/a3.twimg.com\/profile_background_images\/192936036\/TunisiaFlag.jpg","friends_count":187,"description":"Sultan Sooud Al Qassemi is a columnist for The National.","statuses_count":9767,"favourites_count":296,"time_zone":"Abu Dhabi","profile_text_color":"333333","protected":false,"url":null,"profile_image_url":"http:\/\/a3.twimg.com\/profile_images\/1216179693\/FreeTunisia_normal.jpg","notifications":null,"profile_sidebar_fill_color":"efefef","name":"Sultan Al Qassemi","id_str":"46744791","contributors_enabled":false,"lang":"en","profile_background_tile":true,"screen_name":"SultanAlQassemi","id":46744791,"following":null,"verified":false,"utc_offset":14400},"coordinates":null,"retweet_count":6,"truncated":false,"text":"LA Times: Al Jazeera English hopes coverage of Egypt will boost reach in US http:\/\/lat.ms\/gmb15c I salute #AJEnglish's coverage of Egypt","favorited":false,"created_at":"Sat Feb 05 15:11:42 +0000 2011","in_reply_to_user_id":null,"in_reply_to_status_id":null,"source":"web","in_reply_to_screen_name":null,"in_reply_to_status_id_str":null,"entities":{"hashtags":[],"urls":[{"indices":[76,96],"url":"http:\/\/lat.ms\/gmb15c","expanded_url":null}],"user_mentions":[{"indices":[106,116],"name":"Al Jazeera English","screen_name":"AJEnglish","id_str":"4970411","id":4970411}]},"contributors":null,"place":null,"geo":null,"retweeted":false,"in_reply_to_user_id_str":null,"id_str":"33905612630794240","id":33905612630794240},"id":33905938419159040}
{"user":{"statuses_count":774,"profile_link_color":"1581d5","profile_sidebar_border_color":"fac1d8","followers_count":77,"contributors_enabled":false,"profile_use_background_image":true,"created_at":"Tue May 19 19:16:34 +0000 2009","location":"Salina, Kansas","verified":false,"profile_background_color":"b8b7b8","follow_request_sent":null,"profile_background_image_url":"http:\/\/a3.twimg.com\/profile_background_images\/163464585\/twitterbg.jpg","description":"I am a woman of many titles: Mom, college student, daughter, wife, photographer, artist, humanitarian, book addict","is_translator":false,"favourites_count":4,"time_zone":"Central Time (US & Canada)","profile_text_color":"1a1415","protected":false,"url":null,"profile_image_url":"http:\/\/a1.twimg.com\/profile_images\/1151807480\/profile_normal.jpg","notifications":null,"profile_sidebar_fill_color":"f6c1c4","name":"Holly Williams","id_str":"41185779","listed_count":4,"lang":"en","profile_background_tile":true,"screen_name":"hwilliams83","id":41185779,"show_all_inline_media":false,"following":null,"geo_enabled":true,"utc_offset":-21600,"friends_count":158},"coordinates":null,"retweet_count":32,"truncated":false,"text":"RT #petapixel: Egypt protests turn violent: http:\/\/j.mp\/gfErjh","favorited":false,"created_at":"Sat Feb 05 15:13:00 +0000 2011","in_reply_to_user_id":null,"in_reply_to_status_id":null,"source":"web","in_reply_to_screen_name":null,"in_reply_to_status_id_str":null,"entities":{"hashtags":[],"urls":[{"indices":[44,62],"url":"http:\/\/j.mp\/gfErjh","expanded_url":null}],"user_mentions":[{"indices":[3,13],"name":"PetaPixel","screen_name":"petapixel","id_str":"37971731","id":37971731}]},"contributors":null,"place":null,"geo":null,"retweeted":false,"in_reply_to_user_id_str":null,"id_str":"33905940545671169","retweeted_status":{"user":{"statuses_count":4938,"profile_link_color":"2855e9","profile_sidebar_border_color":"dfe3ed","followers_count":80064,"contributors_enabled":false,"profile_use_background_image":true,"created_at":"Tue May 05 17:28:51 +0000 2009","location":"Berkeley, California","verified":false,"profile_background_color":"ffffff","follow_request_sent":null,"profile_background_image_url":"http:\/\/a3.twimg.com\/profile_background_images\/22970429\/twitterbg.jpg","description":"A photography blog for the Web 2.0 generation. We also tweet the tastiest photo and art related links on the web!","is_translator":false,"favourites_count":0,"time_zone":"Pacific Time (US & Canada)","profile_text_color":"000000","protected":false,"url":"http:\/\/petapixel.com","profile_image_url":"http:\/\/a0.twimg.com\/profile_images\/211156611\/avatar_normal.jpg","notifications":null,"profile_sidebar_fill_color":"f5f5f5","name":"PetaPixel","id_str":"37971731","listed_count":10367,"lang":"en","profile_background_tile":false,"screen_name":"petapixel","id":37971731,"show_all_inline_media":true,"following":null,"geo_enabled":false,"utc_offset":-28800,"friends_count":0},"coordinates":null,"retweet_count":32,"truncated":false,"text":"Egypt protests turn violent: http:\/\/j.mp\/gfErjh","favorited":false,"created_at":"Sat Feb 05 13:05:02 +0000 2011","in_reply_to_user_id":null,"in_reply_to_status_id":null,"source":"\u003Ca href=\"http:\/\/www.petapixel.com\" rel=\"nofollow\"\u003EPetaPixel\u003C\/a\u003E","in_reply_to_screen_name":null,"in_reply_to_status_id_str":null,"entities":{"hashtags":[],"urls":[{"indices":[29,47],"url":"http:\/\/j.mp\/gfErjh","expanded_url":null}],"user_mentions":[]},"contributors":null,"place":null,"geo":null,"retweeted":false,"in_reply_to_user_id_str":null,"id_str":"33873733726765056","id":33873733726765056},"id":33905940545671169}
Here is what I think would work better (unfortunately I don't control what twitter sends :
[
{"user":{"statuses_count":357,"profile_link_color":"93A644","profile_sidebar_border_color":"eeeeee","followers_count":28,"contributors_enabled":false,"profile_use_background_image":true,"created_at":"Fri Jun 04 22:45:23 +0000 2010","location":"Brazil","verified":false,"profile_background_color":"B2DFDA","follow_request_sent":null,"profile_background_image_url":"http:\/\/a3.twimg.com\/profile_background_images\/201416971\/egito.jpg","description":"Dare to defy the status quo!!! For a free Egypt!","is_translator":false,"favourites_count":14,"time_zone":"Brasilia","profile_text_color":"333333","protected":false,"url":null,"profile_image_url":"http:\/\/a2.twimg.com\/profile_images\/1233185161\/Flag-Pins-Brazil-Egypt_normal.jpg","notifications":null,"profile_sidebar_fill_color":"ffffff","name":"lu costa","id_str":"152038309","listed_count":0,"lang":"en","profile_background_tile":true,"screen_name":"dacostalu","id":152038309,"show_all_inline_media":false,"following":null,"geo_enabled":false,"utc_offset":-10800,"friends_count":35},"coordinates":null,"retweet_count":49,"truncated":false,"text":"RT #arwasm: Protesters are calling for volunteers to be at the entrances to #tahrir. Entrances are more exposed than yesterday. #jan25 # ...","favorited":false,"created_at":"Sat Feb 05 15:13:00 +0000 2011","in_reply_to_user_id":null,"in_reply_to_status_id":null,"source":"web","in_reply_to_screen_name":null,"in_reply_to_status_id_str":null,"entities":{"hashtags":[{"indices":[76,83],"text":"tahrir"},{"indices":[128,134],"text":"jan25"}],"user_mentions":[{"indices":[3,10],"name":"Arwa Mahmoud","screen_name":"arwasm","id_str":"63693027","id":63693027}],"urls":[]},"contributors":null,"place":null,"geo":null,"retweeted":false,"in_reply_to_user_id_str":null,"id_str":"33905938754834432","retweeted_status":{"user":{"statuses_count":3057,"profile_link_color":"990000","profile_sidebar_border_color":"DFDFDF","followers_count":1468,"contributors_enabled":false,"profile_use_background_image":true,"created_at":"Fri Aug 07 11:21:27 +0000 2009","location":"Cairo, Egypt","verified":false,"profile_background_color":"EBEBEB","follow_request_sent":null,"profile_background_image_url":"http:\/\/a2.twimg.com\/profile_background_images\/70305782\/even_smaller.jpg","description":"Writer, blogger, mountaineer wannabe. Author of A Battle of Creed & \u0642\u062a\u0627\u0644 \u062d\u0632\u0628 \u0627\u0644\u0644\u0647. Falls head over heals with every mountain she meets!","is_translator":false,"favourites_count":0,"time_zone":"Santiago","profile_text_color":"333333","protected":false,"url":"http:\/\/www.alhurr.com","profile_image_url":"http:\/\/a0.twimg.com\/profile_images\/1233059668\/e296709a-cb86-45fd-aa75-89d3fa8a02ac_normal.png","notifications":null,"profile_sidebar_fill_color":"F3F3F3","name":"Arwa Mahmoud","id_str":"63693027","listed_count":156,"lang":"en","profile_background_tile":true,"screen_name":"arwasm","id":63693027,"show_all_inline_media":false,"following":null,"geo_enabled":false,"utc_offset":-14400,"friends_count":182},"coordinates":null,"retweet_count":49,"truncated":false,"text":"Protesters are calling for volunteers to be at the entrances to #tahrir. Entrances are more exposed than yesterday. #jan25 #egypt","favorited":false,"created_at":"Sat Feb 05 14:40:39 +0000 2011","in_reply_to_user_id":null,"in_reply_to_status_id":null,"source":"\u003Ca href=\"http:\/\/blackberry.com\/twitter\" rel=\"nofollow\"\u003ETwitter for BlackBerry\u00ae\u003C\/a\u003E","in_reply_to_screen_name":null,"in_reply_to_status_id_str":null,"entities":{"hashtags":[{"indices":[64,71],"text":"tahrir"},{"indices":[116,122],"text":"jan25"},{"indices":[123,129],"text":"egypt"}],"user_mentions":[],"urls":[]},"contributors":null,"place":null,"geo":null,"retweeted":false,"in_reply_to_user_id_str":null,"id_str":"33897797342199808","id":33897797342199808},"id":33905938754834432},
{"user":{"follow_request_sent":null,"profile_link_color":"0099B9","is_translator":false,"profile_sidebar_border_color":"5ED4DC","followers_count":528,"profile_use_background_image":true,"created_at":"Sun Mar 08 04:52:10 +0000 2009","location":"SE USA","listed_count":7,"profile_background_color":"0099B9","show_all_inline_media":false,"geo_enabled":false,"profile_background_image_url":"http:\/\/a0.twimg.com\/profile_background_images\/17691490\/Bloom.jpg","friends_count":550,"description":"tech, motorcycles, photography, music, green, generally curious","statuses_count":6008,"favourites_count":95,"time_zone":"Central Time (US & Canada)","profile_text_color":"3C3940","protected":false,"url":null,"profile_image_url":"http:\/\/a3.twimg.com\/profile_images\/287875274\/Speedo_normal.jpg","notifications":null,"profile_sidebar_fill_color":"95E8EC","name":"Henk Sijgers","id_str":"23277821","contributors_enabled":false,"lang":"en","profile_background_tile":false,"screen_name":"henksijgers","id":23277821,"following":null,"verified":false,"utc_offset":-21600},"coordinates":null,"retweet_count":6,"truncated":true,"text":"RT #SultanAlQassemi: LA Times: Al Jazeera English hopes coverage of Egypt will boost reach in US http:\/\/lat.ms\/gmb15c I salute #AJEnglis ...","favorited":false,"created_at":"Sat Feb 05 15:13:00 +0000 2011","in_reply_to_user_id":null,"in_reply_to_status_id":null,"source":"\u003Ca href=\"http:\/\/itunes.apple.com\/app\/twitter\/id333903271?mt=8\" rel=\"nofollow\"\u003ETwitter for iPad\u003C\/a\u003E","in_reply_to_screen_name":null,"in_reply_to_status_id_str":null,"entities":{"hashtags":[],"urls":[{"indices":[97,117],"url":"http:\/\/lat.ms\/gmb15c","expanded_url":null}],"user_mentions":[{"indices":[3,19],"name":"Sultan Al Qassemi","screen_name":"SultanAlQassemi","id_str":"46744791","id":46744791},{"indices":[127,136],"name":"Mahandini Teukie","screen_name":"ajenglis","id_str":"241013489","id":241013489}]},"contributors":null,"place":null,"geo":null,"retweeted":false,"in_reply_to_user_id_str":null,"id_str":"33905938419159040","retweeted_status":{"user":{"follow_request_sent":null,"profile_link_color":"009999","is_translator":false,"profile_sidebar_border_color":"eeeeee","followers_count":27255,"profile_use_background_image":true,"created_at":"Fri Jun 12 20:47:01 +0000 2009","location":"\u00dcT: 25.206282,55.281949","listed_count":1794,"profile_background_color":"131516","show_all_inline_media":true,"geo_enabled":false,"profile_background_image_url":"http:\/\/a3.twimg.com\/profile_background_images\/192936036\/TunisiaFlag.jpg","friends_count":187,"description":"Sultan Sooud Al Qassemi is a columnist for The National.","statuses_count":9767,"favourites_count":296,"time_zone":"Abu Dhabi","profile_text_color":"333333","protected":false,"url":null,"profile_image_url":"http:\/\/a3.twimg.com\/profile_images\/1216179693\/FreeTunisia_normal.jpg","notifications":null,"profile_sidebar_fill_color":"efefef","name":"Sultan Al Qassemi","id_str":"46744791","contributors_enabled":false,"lang":"en","profile_background_tile":true,"screen_name":"SultanAlQassemi","id":46744791,"following":null,"verified":false,"utc_offset":14400},"coordinates":null,"retweet_count":6,"truncated":false,"text":"LA Times: Al Jazeera English hopes coverage of Egypt will boost reach in US http:\/\/lat.ms\/gmb15c I salute #AJEnglish's coverage of Egypt","favorited":false,"created_at":"Sat Feb 05 15:11:42 +0000 2011","in_reply_to_user_id":null,"in_reply_to_status_id":null,"source":"web","in_reply_to_screen_name":null,"in_reply_to_status_id_str":null,"entities":{"hashtags":[],"urls":[{"indices":[76,96],"url":"http:\/\/lat.ms\/gmb15c","expanded_url":null}],"user_mentions":[{"indices":[106,116],"name":"Al Jazeera English","screen_name":"AJEnglish","id_str":"4970411","id":4970411}]},"contributors":null,"place":null,"geo":null,"retweeted":false,"in_reply_to_user_id_str":null,"id_str":"33905612630794240","id":33905612630794240},"id":33905938419159040},
{"user":{"statuses_count":774,"profile_link_color":"1581d5","profile_sidebar_border_color":"fac1d8","followers_count":77,"contributors_enabled":false,"profile_use_background_image":true,"created_at":"Tue May 19 19:16:34 +0000 2009","location":"Salina, Kansas","verified":false,"profile_background_color":"b8b7b8","follow_request_sent":null,"profile_background_image_url":"http:\/\/a3.twimg.com\/profile_background_images\/163464585\/twitterbg.jpg","description":"I am a woman of many titles: Mom, college student, daughter, wife, photographer, artist, humanitarian, book addict","is_translator":false,"favourites_count":4,"time_zone":"Central Time (US & Canada)","profile_text_color":"1a1415","protected":false,"url":null,"profile_image_url":"http:\/\/a1.twimg.com\/profile_images\/1151807480\/profile_normal.jpg","notifications":null,"profile_sidebar_fill_color":"f6c1c4","name":"Holly Williams","id_str":"41185779","listed_count":4,"lang":"en","profile_background_tile":true,"screen_name":"hwilliams83","id":41185779,"show_all_inline_media":false,"following":null,"geo_enabled":true,"utc_offset":-21600,"friends_count":158},"coordinates":null,"retweet_count":32,"truncated":false,"text":"RT #petapixel: Egypt protests turn violent: http:\/\/j.mp\/gfErjh","favorited":false,"created_at":"Sat Feb 05 15:13:00 +0000 2011","in_reply_to_user_id":null,"in_reply_to_status_id":null,"source":"web","in_reply_to_screen_name":null,"in_reply_to_status_id_str":null,"entities":{"hashtags":[],"urls":[{"indices":[44,62],"url":"http:\/\/j.mp\/gfErjh","expanded_url":null}],"user_mentions":[{"indices":[3,13],"name":"PetaPixel","screen_name":"petapixel","id_str":"37971731","id":37971731}]},"contributors":null,"place":null,"geo":null,"retweeted":false,"in_reply_to_user_id_str":null,"id_str":"33905940545671169","retweeted_status":{"user":{"statuses_count":4938,"profile_link_color":"2855e9","profile_sidebar_border_color":"dfe3ed","followers_count":80064,"contributors_enabled":false,"profile_use_background_image":true,"created_at":"Tue May 05 17:28:51 +0000 2009","location":"Berkeley, California","verified":false,"profile_background_color":"ffffff","follow_request_sent":null,"profile_background_image_url":"http:\/\/a3.twimg.com\/profile_background_images\/22970429\/twitterbg.jpg","description":"A photography blog for the Web 2.0 generation. We also tweet the tastiest photo and art related links on the web!","is_translator":false,"favourites_count":0,"time_zone":"Pacific Time (US & Canada)","profile_text_color":"000000","protected":false,"url":"http:\/\/petapixel.com","profile_image_url":"http:\/\/a0.twimg.com\/profile_images\/211156611\/avatar_normal.jpg","notifications":null,"profile_sidebar_fill_color":"f5f5f5","name":"PetaPixel","id_str":"37971731","listed_count":10367,"lang":"en","profile_background_tile":false,"screen_name":"petapixel","id":37971731,"show_all_inline_media":true,"following":null,"geo_enabled":false,"utc_offset":-28800,"friends_count":0},"coordinates":null,"retweet_count":32,"truncated":false,"text":"Egypt protests turn violent: http:\/\/j.mp\/gfErjh","favorited":false,"created_at":"Sat Feb 05 13:05:02 +0000 2011","in_reply_to_user_id":null,"in_reply_to_status_id":null,"source":"\u003Ca href=\"http:\/\/www.petapixel.com\" rel=\"nofollow\"\u003EPetaPixel\u003C\/a\u003E","in_reply_to_screen_name":null,"in_reply_to_status_id_str":null,"entities":{"hashtags":[],"urls":[{"indices":[29,47],"url":"http:\/\/j.mp\/gfErjh","expanded_url":null}],"user_mentions":[]},"contributors":null,"place":null,"geo":null,"retweeted":false,"in_reply_to_user_id_str":null,"id_str":"33873733726765056","id":33873733726765056},"id":33905940545671169}
]

This was quite simple actually... I just needed to check if the data ends with \r\n and then store the results of my parser before releasing it and creating a new one.