What's the difference between the Perl JSON modules below?
I have come across JSON::PP and JSON::XS. The documentation of JSON::PP says it is compatible with JSON::XS. What does that mean?
I am not sure what the difference between them are, let alone which of them to use. Can someone clarify?
Perl modules sometimes have different implementations. The ::PP suffix is for the Pure Perl implementation (i.e. for portability), the ::XS suffix is for the C-based implementation (i.e. for speed), and JSON is just the top-level module itself (i.e. the one you actually use).
As noted by #Quentin, this site has a good description of them. To quote:
JSON
JSON.pm is a wrapper around JSON::PP and JSON::XS - it also does a bunch of moderately crazy things for compatibility reasons, including extra shim code for very old perls [...]
JSON::PP
This is the standard pure perl implementation, and if you're not performance dependent, there's nothing wrong with using it directly [...]
JSON::XS
Ridiculously fast JSON implementation in C. Absolutely wonderful [...]
As you can see, just installing the top-level JSON module should do it for you. The part about compatibility just means that they both do the same thing, i.e. you should get the same output from both.
I installed the Perl JSON module a few years ago on a RHEL server I managed and it was a really straightforward process: just install (or build) the module from the CPAN site and you're done.
Installing should be a simple case of either using the OS package manager (if in GNU/Linux), using the cpan utility, or building from source. The OS package manager is recommended, as it helps keep things updated automatically.
To verify that it's installed, just try the following command from the terminal (assuming GNU/Linux):
$ perl -e 'use JSON;'
If it doesn't complain, then you should be good to go. If you get errors, then you should get ready to go in an adventure.
You can install JSON module, cpan install JSON
use JSON;
my $result = from_json($json);
if($result->{field})
{
# YOUR CODE
};
Related
This question already has answers here:
Executing system commands safely while coding in Perl
(3 answers)
Closed 6 years ago.
Like executing a command using backticks or exec() or system().
Yes, this is bad practice in almost all cases! This is especially true if you have a choice.
Executing external commands is:
Very hard to do correctly (but very easy to get sort-of-working). You can't just escape the shell args and call it a day, you have to account for potentially multiple levels of escaping for potentially different languages.
Slow. A fork+exec to e.g. rm is easily a thousand times slower than the corresponding syscall.
A rigid, error-prone and inexpressive integration point. You typically have to convert data to flat lists of strings and back. You can't use the language's features like exception handling, nested data structures or callbacks.
Due to this, the following are BAD reasons to call external commands:
Not knowing how to do X in your language, but knowing a shell command for it. A typical example is cp -R foo bar.
Not knowing how something works, but knowing a shell oneliner that does it. A typical example is foo *.mp4 > >(tee file).
Not wanting to learn a new API for e.g. json or http, and instead using shell tools like jq or curl.
However, if you are calling a program that does non-trivial things, that doesn't have a native library or bindings, AND that you know how to invoke with execve semantics (NOT system nor perl exec semantics that invoke shells), this is a valuable tool.
Examples of good uses of executing external commands that follow all the above is invoking make to build a project from an installer, or running java -jar ... to start a Minecraft server.
It's a great* practice. Perl and PHP are great for lots of things, but they're not great at everything, and there's a use case for using external programs and other tools in your project. But one of things that Perl is definitely great at is gluing together input and output formats and letting you mash together several different tools into a single project, letting each part of the project do what they do best.
* by which I mean, often a great practice. Things like #files=qx(ls $dir) and #txt=qx(cat $textfile) make all right-thinking Perl programmers cringe.
Some operating system commands might have built-in functionality not available in language (Perl's mkdir() lacks *ix mkdir's -p). Then again, something might be easier to do using languages constructs instead of parsing output (readdir() vs ls).
And it is important to remember that something written in Perl might be more portable to non-Unix systems than calling OS-specific external programs.
Why not?
But, be careful and escape all strings inserted in the command:
In PHP: escapeshellarg()
In Perl: Perl equivalent of PHP's escapeshellarg
I've been an avid user of CGI.pm since the previous millennium so I was a bit surprised when it disappeared from my old Ubuntu server when I upgraded it recently. My short-term fix was sudo cpan install CGI, but a quick web search to find out why it was missing in the first place revealed CGI::Alternatives which explains why it has gone and offers some suggestions for alternatives. For my purposes, HTML::Tiny looks best for replacing my programmatic HTML generation, but Alternatives is strangely silent on the subject of HTTP headers and CGI parameters.
I broadened by search and found lighter alternatives to CGI.pm on perlmonks where one response suggests CGI::Simple, but the recommendation is less than whole-hearted - "its not quite as up to date as CGI.pm".
So is CGI::Simple the way to go, or is there a better alternative?
Please don't spend time suggesting "rewrite everything using framework XXX". I really don't have the time or energy for that. I'm happy to replace all my HTML generation with HTML::Tiny, so I'm looking for something with a similar (or lower!) amount of rework to replace header() and param().
You're missing the point if you're looking for an alternative that provides header and param.
The argument for the removal of CGI.pm from core (but not from CPAN) is that you shouldn't have to deal with CGI yourself; you should be using a framework that handles this for you.
If you don't agree with this — if you're looking for an equivalent to header and param — go ahead and keep using CGI.pm.
If you do agree, CGI::Simple is no better than CGI.pm.
As others have said, there's no reason not to use CGI together with HTML::Tiny. So that's the answer to your question. For the last five years that I was using CGI, my programs all started something like:
use CGI qw[param header];
which is the approach you're talking about here.
If you wait a year or two, the plan is for the HTML generation functions to be removed from the main module, so your problems will all go away at that point.
But that's not what I'd do in your situation. I'd switch to using PSGI and Plack. You said that you don't want anyone to suggest a new framework, so I'm not going to do that. Plack isn't a framework, it's a toolbox for writing PSGI applications. Certainly, I'd use a framework like Dancer, but you don't have to. You can happily use Plack without any of the frameworks built on top of it.
You'll still get most of the advantages of PSGI. You'll be able to deploy your applications in any way you like. You'll have access to all the awesome Plack middleware. Testing your program will be far easier.
When you're using "raw" Plack, the equivalent of CGI::param is Plack::Request::parameters and the equivalent of CGI::header is Plack::Response::headers.
So there are three answers to your question.
Carry on using CGI.pm. Just stop using the HTML generation functions and replace them with HTML::Tiny
Use raw PSGI/Plack and bring your web development into the 21st century
Use one of Perl's many great web frameworks.
Unfortunately, you don't seem to like any of those answers.
The issue with CGI.pm is not that it's going away, merely that it will no longer be distributed as part of the core Perl distribution. However that doesn't mean you have to install from CPAN. On your Ubuntu system you can just do:
sudo apt-get install libcgi-pm-perl
and you'll be off and running with the same old CGI you know and love :-)
The correct answer to my question is that use CGI::Simple is better than use CGI qw(header param) because it loads faster.
Answers along the lines of "Use Plack, it's the future of Perl for websites" weren't helpful to me because I didn't have time to learn a new programming paradigm or to discover how to reconfigure my web server to make it work, no matter how insistent the Plack Evangelists were that I was wrong in what I was trying to do.
I've now had a bit of time to wade through the links to documentation and presentation slides I was offered and I can see what they were getting at, but one failing in what I've read so far is the lack of a concise end-to-end working example to help get my head around things ... so here's what I knocked together to get me started (and, no, I haven't finished yet!). I hope that others beginning the journey from CGI to PSGI will find this useful to help get them underway...
First you need to install Plack. I'm running an Ubuntu 14.04 installation so it was simply a matter of running sudo apt-get install libplack-perl. The generic way is to install Task::Plack from CPAN.
Next you need to know where your cgi-bin directory is located. You ought to know already if you're a CGI die-hard! Since I'm running Apache mine is defined in /etc/apache2/conf-available/serve-cgi-bin.conf by ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/.
Now for the magic. We're going to create a CGI script that runs a PSGI app, handing it data from the CGI environment. This is good for experimentation and testing but NOT for deployment, as you don't get any of the speed benefit that PSGI can give you (for that you need something like Plack::Handler::Apache2, Plack::Handler::FCGI or mod_psgi in Apache, or a dedicated PSGI server such as Starman or Starlet, or one of the other handlers mentioned on PlackPerl.org). Create /usr/lib/cgi-bin/psgi-cgi.pl with the following contents and make it executable:-
#!/usr/bin/perl
use Plack::Util;
use Plack::Handler::CGI;
my $app = Plack::Util::load_psgi($ENV{PATH_TRANSLATED});
Plack::Handler::CGI->new->run($app);
Next we need to tell Apache to pass PSGI app files to this handler. I did this by creating /etc/apache2/conf-available/psgi-cgi.conf containing:-
Action psgi-cgi /cgi-bin/psgi-cgi.pl
AddHandler psgi-cgi .psgi
then loaded it into my Apache server by running sudo a2enconf psgi-cgi and sudo service apache2 reload. Basically you need to get these lines into your httpd.conf file and restart the server.
Finally, my first PSGI script, which I created in my server's DocumentRoot as /var/www/html/hello.psgi:-
use Plack::Request;
my $app = sub {
my $env = shift;
my $req = Plack::Request->new($env);
my $par = $req->parameters;
return [
200,
[ 'Content-Type', 'text/plain' ],
[ "Hello world!\n",
map("$_ = ".join(", ", $par->get_all($_))."\n", sort keys %$par),
]
];
};
The application is a coderef which returns a 3-element arrayref: the first is the HTTP status code, the second is the name,value pairs for the HTTP header, the third is the body of the response (which could be generated using HTML::Tiny for a web page). The first 2 elements answer the question of what you need instead of the CGI::header function - nothing! (though for more complex handling you'll need Plack::Response::headers). The example also shows how to replace CGI::param - use Plack::Request::parameters, which returns a Hash::MultiValue object containing the values of URL (GET) and BODY (POST) parameters, including the ones with multiple values.
Finally, a test:-
$ wget -q -O- 'http://localhost/hello.psgi?a=1&a=2&a=3&b=1&b=4'
Hello world!
a = 1, 2, 3
b = 1, 4
I hope this is useful to other CGI die-hards in taking their first steps towards PSGI proficiency, and I hope the Plack Evangelists will acknowledge that it takes a lot of reading and comprehension to get even this far.
CGI::Minimal would a good option, it is much lighter than CGI & CGI::Simple, but it lacks advanced methods like CGI & CGI::Simple
When attempting to use pandoc to convert JSON based files (.ipynb) from iPython notebook (0.12), I receive an error stating "bad decodeArgs" for the JSON. I suspect that it may be due to the Ubuntu provided version of pandoc that I am using (1.8.1.1). It seems that getting the latest pandoc version requires setting up the Haskell platform which I was not successful doing because of dependency challenges (and really don't want to). I don't want to spend any more time trying to install Haskell if this is not my problem.
Is there a way to get the latest pandoc binaries for Ubuntu without rebuilding it?
Given that iPython notebook is new (and very cool!!), it would be nice to hear about experiences related to translating the JSON to other formats. Perhaps there is a different way to accomplish this other than pandoc.
Regarding your "keeping up to date with Pandoc", I'm afraid you do need Haskell installed. The best way to do this via the Haskell Platform ("HP") package, and then just like with Ruby, it's a lot more consistent to use the environment's package manager for dependencies than your OS. I've had no trouble getting it working, even in Windoze. . .
I'm sure questions to the Haskell mailing list would result in quick help for a platform as mainstream as Debian/Ubuntu, but you might need to manually install a newer version of HP that what's available through the OS package manager.
Once you get HP up and running, the dev Pandoc is dead easy to compile, and git will keep you up to date with the latest - specific instructions here, currently maintained:
https://github.com/jgm/pandoc/wiki/Installing-the-development-version-of-pandoc-1.9
Note that v1.9 has now been officially released if you really don't want to go to the trouble of keeping up to date with the dev cycle, but of course again you won't get it in your OS package manager for quite some time after that (I assume anyway).
==========================
Regarding your attempts to treat JSON as a document syntax:
The best syntax inputs for Pandoc at this point are its native markdown+extensions, and reST (especially for Python people/environments), basically maintained as functionally equivalent, although there may be features available in the former that aren't represented in the latter, since John can just add extensions anytime he wants. AFAIK Pandoc hasn't begun to support the Sphinx extensions (yet?)
The JSON format used internally within Pandoc isn't documented (yet?) but it's the native Haskell data type. As the Thomas K notes, there may be some similarity between how the two tools represent data, but probably not enough to treat either as "just another markup format".
However, if you're working on this, it's easy enough to see what Pandoc looks for in the way of JSON input.
pandoc -t json
compare this to
pandoc -t native
and it's easy to see the specs created by Text.Pandoc.Definition and Text.JSON.Generic
Using Pandoc's internal data representation as input would obviously be more stable than a marked up text stream, and others have expressed a desire for documentation on this and it would be a great contribution to the community.
Please do inform the Pandoc mail list of any work done in this area. The crew there is very responsive, including getting quick feedback from John M (the lead developer) himself directly.
I doubt pandoc or any other tool knows what to do with ipynb files yet (at the time of writing, the IPython notebook was released less than a month ago). JSON is just a generic data structure like XML, not a document format.
We're (IPython) working on tools to export notebooks to other formats, but they're not ready for a proper release yet. If you want to help develop that, see this mailing list thread. Hopefully it will be part of the next IPython release.
I've installed the Yojson library for OCaml via GODI:
http://martin.jambon.free.fr/yojson.html
I want to start an interactive ocaml session (i.e. via the ocaml command) and execute functions from the Yojson library e.g.
Yojson.Safe.from_string;;
How do I do this? The above command gives "Error: Unbound module Yojson". I've worked out how to compile via ocamlc with Yojson available, but I want to launch an interactive session instead.
I know this seems like a horrible beginners question but Yojson comes with no samples and minimal instructions so I'm really stumped. I've tried various combinations of "#load" and compiler switches and I'm stuck.
The tool you are after is called findlib. It is included in the base GODI installation. The tools that come with findlib allow you to easily compile against most OCaml libraries and use those libraries from a toplevel session (ocaml). The findlib documentation is fairly comprehensive, but here is a quick summary to get started.
To start using findlib from within a toplevel session:
#use "topfind";;
This will display a brief usage message. Then you can type:
#list;;
This will show you a list of all of the available packages. Yojson will likely be among them. Finally:
#require "yojson";;
where yojson is replaced by the appropriate entry shown by #list;;. Yojson's modules should be available for you to use at this point.
Are there any software packages or projects that provide the scripting language shells? I know there's csh for C programmers although not in a sense that it's primarily for programming, but for navigation and system administration. I was wondering if there is something inverted for this purpose? I.e. user logs into a shell that's primarily for programming and then for navigation (something like irb in ruby, but with navigation capabilities)?
I think you're misinformed if you think csh (tcsh) is for C programmers. It's just a shell like bash or ash or dash or ksh or zsh.
The R language provides a reasonably functional internal environment, complete with the ability to save/restore the "workspace" (your variables).
Python has a built-in interpreter, as does Maxima, and some Lisp/Scheme versions, plus you already mentioned irb.
You could also view vim or emacs as the type of programmer-centric shell you're talking about; both can be hooked up to run navigation commands and sysadmin-type stuff without forcing you to leave the editor.
I think the real answer to your question is "powerful shells provide their own scripting language".
Tcl's interpreter, tclsh, is really designed to be a shell. In fact, unlike Ruby where the interactive and non interactive shell are separate, tclsh works just like traditional shells like bash: if run without a script it enters interactive mode but given a script it enters batch mode.
But, it does suck in that it doesn't have readline built-in. So no up-arrow history or tab completion etc. But you can always run it using rlwrap:
rlwrap tclsh
which should give you readline capabilities.
However, I wasn't satisfied (partly because my system at the time didn't have rlwrap and partly because there were a few more features I wanted). So I wrote my own implementation of history and tab completion etc. Checkout my original Pure-tcl readline or the improved Pure-tcl readline2.
It really does act like an interactive shell complete with auto-executing external executables if a tcl command is unknown. And you can even execute interactive programs like vi, emacs or lynx from it. Because it automatically falls back to executing external commands, you can mix tcl and shell like:
foreach x [split [ps aux | grep apache] \n] {
puts [lindex $x 1]
}
This is great because tcl's syntax is much saner compared to bash and sh (ever tried to get out of '"\"\\\\"\ quoting hell in bash?). I personally like tcl but tcl is kind of a love-it-or-hate-it language. People who get it really love it and people who don't really hate it.
But even if you don't quite like tcl syntax I'd suggest you give it a try for this specific application because unlike other languages tcl really is designed to be used more as a command language than a programming language. Read I can’t believe I’m praising Tcl for some of the reasons why.
System navigation (and administrative tasks) are a really different application than programming, and it's hard to find a single shell that does both well. However, I'm guessing that what you're really asking for is a shell that
Lets you easily load the contents of a file and manipulate those contents in-process and with more dexterity than you get using bash and standard unix utilities.
In addition you want the convenience of accessing some of the normal commands for moving files around and navigating the file system.
The good news is that the standard scripting languages (e.g. Ruby, Perl) were meant to do #1 really well, and it's not hard to write/find a library to do #2 any of these langauges.
Because Ruby is what I'm familiar with, I'm going to give you a more concrete example of how you might accomplish this using Ruby.
To do this in Ruby, you would use irb (the Ruby REPL), and the FileUtils module which is part of Ruby's standard library.
To do this, start irb, then run
require 'fileutils'
include FileUtils
(you can put this in .irbrc if you'd like, but I'm not sure I'd recommend that.)
this allows you to have access to a number of the normal file manipulation commands through easy Ruby syntax. You can run other Ruby commands automatically yourself. To run other commands on your system, you're going to have to call them with system.
FileUtils doesn't include an ls command, because it wasn't really meant to be used interactively, so you'll need to write your own. I don't know a way to get good job control at all (that's not to say you couldn't write something though).
The only thing I warn you is that this workflow will be very different than other UNIX users, so you might want to think about being such a nonconformist is worth it, or whether you'd rather build experience that meshes well with other UNIX admins' working styles. It's probably better to get used to the core UNIX utils and the Bourne shell scripting language. (You could learn C-Shell if you want, but there is a well-known FAQ explaining the disadvantages of the C-shell for programming.)
You may want to take a look at IPython. It is an interactive Python shell (with filesystem navigation alongside other nice features) and it also provides a system shell profile to optimize its behavior for system shell usage.
CSH has nothing to do with C programming. It's serves the same functions as the Bourne Shell about equally well, but uses different syntax.
If you want C interpreter, I suggest using cint, which is part of CERN's ROOT system. But keep in mind that it's not useful in the least for system administration and navigation.
I'm sure with a little bit of work you could further extend Devel::REPL (Perl) to provide access to gnu coreutils,
Bash has lots of programming features that aren't ordinarily acknowledged, for example arrays and string manipulation options when expanding a shell variable. Some shells, like zsh or ksh have greatly improved programming features compared to the most common shells (namely bash or tcsh.)