Compress JSON file by eliminating whitespace - json

Question

I am working with a large json file (~100,000 lines) and need to compress it down to make a program run faster. I wish to delete all the horizontal tabs, returns, etc. to minimize the size of the file.
For example if a line was originally:
"name_id": "Richard Feynman",
"occupation": "Professional Bongos Player"
it should be compressed to:
"name_id":"Richard Feynman","occupation":"Professional Bongos Player"`
I have scoured the Internet (forgive me if it is a simple answer, I am a beginner) and can't seem to find a command for the terminal that will help me do this. Any help would be much appreciated

Answer 1

Looks like you're looking for a JSON minifier.
There are some around, both online and standalone.
Try googling these terms + your favorite language, I'm sure you'll find something that suits your needs.
There are other tools that modify your JSON to make it smaller, but you'll end up with a different JSON, I guess. Haven't tried those.

Answer 2

Using GNU awk for RT:
$ awk 'BEGIN{RS="\""} NR%2{gsub(/[[:space:]]/,"")} {ORS=RT;print} END{printf "\n"}' file
"name_id":"Richard Feynman","occupation":"Professional Bongos Player"

Answer 3

The following flex(1) program will do the work. It makes a lexical analisys of json source and eliminates comments and spaces between tokens, respecting the in-string spaces. It also recognizes unquoted identifiers, and quotes them.
To compile it, just do
make json
Use it with the following command:
json [ file ... ]
if you don't specify a file, the program will read from stdin.
Here's the source:
%{
/* json-min. JSON minimizer.
* Author: Luis Colorado <lc#luiscoloradosistemas.com>
* Date: Wed Aug 13 07:35:23 EEST 2014
* Disclaimer: This program is GPL, as of GPL version 3, you
* may have received a copy of that document, or you can
* instead look at http://www.gnu.org/licenses/gpl.txt to read
* it. There's no warranty, nor assumed nor implicit on the
* use of this program, you receive it `as is' so whatever you
* do with it is only your responsibility. Luis Colorado
* won't assume any responsibility of the use or misuse of
* this program. You are warned.
*/
%}
dec ([1-9][0-9]*)
oct (0[0-7]*)
hex (0[xX][0-9a-fA-F]*)
doub ({dec}"."([0-9]*)?|{dec}?"."[0-9]+)
strd (\"([^\"]|\\.)*\")
t "true"
f "false"
n "null"
com1 "//".*
com2b "/*"
endc "*/"
ident ([a-zA-Z_][a-zA-Z0-9_]*)
%x INCOMMENT
%option noyywrap
%%
{dec} |
{oct} |
{hex} |
{doub} |
{strd} |
{t} |
{f} |
{n} |
"{" |
":" |
";" |
"}" |
"[" |
"]" |
"," ECHO;
[\ \t\n] |
{com1} ;
{com2b} BEGIN(INCOMMENT);
<INCOMMENT>. ;
<INCOMMENT>{endc} BEGIN(INITIAL);
{ident} { fprintf(stderr, "WARNING:"
"unquoted identifier %s "
"in source. Quoting.\n",
yytext);
printf("\"%s\"", yytext);
}
. { fprintf(stderr,
"WARNING: unknown symbol %s "
"in source, copied to output\n",
yytext);
ECHO;
}
%%
void process(const char *fn);
int main(int argc, const char **argv)
{
int i;
if (argc > 1) for (i = 1; i < argc; i++)
process(argv[i]);
else process(NULL); /* <-- stdin */
} /* main */
void process(const char *fn)
{
FILE *f = stdin;
if (fn) {
f = fopen(fn, "r");
if (!f) {
fprintf(stderr,
"ERROR:fopen:%s:%s(errno=%d)\n",
fn, strerror(errno), errno);
exit(EXIT_FAILURE);
} /* if */
} /* if */
yyin = f;
yylex();
if (fn) /* only close if we opened, don't close stdin. */
fclose(f);
printf("\n");
}
I have just written it, so there's little testing on it. Use it with care (conserve a backup of your original file) It doesn't overwrite the original file, just outputs to stdout, so you don't overwrite your data using it.
BR,
Luis