When attempting to use pandoc to convert JSON based files (.ipynb) from iPython notebook (0.12), I receive an error stating "bad decodeArgs" for the JSON. I suspect that it may be due to the Ubuntu provided version of pandoc that I am using (1.8.1.1). It seems that getting the latest pandoc version requires setting up the Haskell platform which I was not successful doing because of dependency challenges (and really don't want to). I don't want to spend any more time trying to install Haskell if this is not my problem.
Is there a way to get the latest pandoc binaries for Ubuntu without rebuilding it?
Given that iPython notebook is new (and very cool!!), it would be nice to hear about experiences related to translating the JSON to other formats. Perhaps there is a different way to accomplish this other than pandoc.
Regarding your "keeping up to date with Pandoc", I'm afraid you do need Haskell installed. The best way to do this via the Haskell Platform ("HP") package, and then just like with Ruby, it's a lot more consistent to use the environment's package manager for dependencies than your OS. I've had no trouble getting it working, even in Windoze. . .
I'm sure questions to the Haskell mailing list would result in quick help for a platform as mainstream as Debian/Ubuntu, but you might need to manually install a newer version of HP that what's available through the OS package manager.
Once you get HP up and running, the dev Pandoc is dead easy to compile, and git will keep you up to date with the latest - specific instructions here, currently maintained:
https://github.com/jgm/pandoc/wiki/Installing-the-development-version-of-pandoc-1.9
Note that v1.9 has now been officially released if you really don't want to go to the trouble of keeping up to date with the dev cycle, but of course again you won't get it in your OS package manager for quite some time after that (I assume anyway).
==========================
Regarding your attempts to treat JSON as a document syntax:
The best syntax inputs for Pandoc at this point are its native markdown+extensions, and reST (especially for Python people/environments), basically maintained as functionally equivalent, although there may be features available in the former that aren't represented in the latter, since John can just add extensions anytime he wants. AFAIK Pandoc hasn't begun to support the Sphinx extensions (yet?)
The JSON format used internally within Pandoc isn't documented (yet?) but it's the native Haskell data type. As the Thomas K notes, there may be some similarity between how the two tools represent data, but probably not enough to treat either as "just another markup format".
However, if you're working on this, it's easy enough to see what Pandoc looks for in the way of JSON input.
pandoc -t json
compare this to
pandoc -t native
and it's easy to see the specs created by Text.Pandoc.Definition and Text.JSON.Generic
Using Pandoc's internal data representation as input would obviously be more stable than a marked up text stream, and others have expressed a desire for documentation on this and it would be a great contribution to the community.
Please do inform the Pandoc mail list of any work done in this area. The crew there is very responsive, including getting quick feedback from John M (the lead developer) himself directly.
I doubt pandoc or any other tool knows what to do with ipynb files yet (at the time of writing, the IPython notebook was released less than a month ago). JSON is just a generic data structure like XML, not a document format.
We're (IPython) working on tools to export notebooks to other formats, but they're not ready for a proper release yet. If you want to help develop that, see this mailing list thread. Hopefully it will be part of the next IPython release.
Related
I have a couple questions about adding options/switches (with and without parameters) to procedures/commands. I see that tcllib has cmdline and Ashok Nadkarni's book on Tcl recommends the parse_args package and states that using Tcl to handle the arguments is much slower than this package using C. The Nov. 2016 paper on parse_args states that Tcl script methods are or can be 50 times slower.
Are Tcl methods really signicantly slower? Is there some minimum threshold number of options to be reached before using a package?
Is there any reason to use parse_args (not in tcllib) over cmdline (in tcllib)?
Can both be easily included in a starkit?
Is this included in 8.7a now? (I'd like to use 8.7a but I'm using Manjaro Linux and am afraid that adding it outside the package manager will cause issues that I won't know how to resolve or even just "undo").
Thank you for considering my questions.
Are Tcl methods really signicantly slower? Is there some minimum threshold number of options to be reached before using a package?
Potentially. Procedures have overhead to do with managing the stack frame and so on, and code implemented in C can avoid a number of overheads due to the way values are managed in current Tcl implementations. The difference is much more profound for numeric code than for string-based code, as the cost of boxing and unboxing numeric values is quite significant (strings are always boxed in all languages).
As for which is the one to use, it really depends on the details as you are trading off flexibility for speed. I've never known it be a problem for command line parsing.
(If you ask me, fifty options isn't really that many, except that it's quite a lot to pass on an actual command line. It might be easier to design a configuration file format — perhaps a simple Tcl script! — and then to just pass the name of that in as the actual argument.)
Is there any reason to use parse_args (not in tcllib) over cmdline (in tcllib)?
Performance? Details of how you describe things to the parser?
Can both be easily included in a starkit?
As long as any C code is built with Tcl stubs enabled (typically not much more than define USE_TCL_STUBS and link against the stub library) then it can go in a starkit as a loadable library. Using the stubbed build means that the compiled code doesn't assume exactly which version of the Tcl library is present or what its path is; those are assumptions that are usually wrong with a starkit.
Tcl-implemented packages can always go in a starkit. Hybrid packages need a little care for their C parts, but are otherwise pretty easy.
Many packages either always build in stubbed mode or have a build configuration option to do so.
Is this included in 8.7a now? (I'd like to use 8.7a but I'm using Manjaro Linux and am afraid that adding it outside the package manager will cause issues that I won't know how to resolve or even just "undo").
We think we're about a month from the feature freeze for 8.7, and builds seem stable in automated testing so the beta phase will probably be fairly short. The list of what's in can be found here (filter for 8.7 and Final). However, bear in mind that we tend to feel that if code can be done in an extension then there's usually no desperate need for it to be in Tcl itself.
Note: this is not a dupe of this or this other question. Read on: this question is specific to the Code-Sharing template.
I am doing some pretty basic experiments with NativeScript, Angular and the code sharing templates (see: #nativescript/schematics).
Now I am doing some exploration / poc work on how different "build configuration" are supported by the framework. To be clear, I am searching for a simple -and hopefully official- way to have the application use a different version of a specific file (let's call it configuration.ts) based on the current platform (web/ios/android) and environment (development/production/staging?).
Doing the first part is obviously trivial - after all that is the prime purpose of the code sharing schematics. So, different versions of the same file are identified by different extensions. This page explain things pretty simply.
What I don't get as easily is if the framework/template supports any similar convention-based rule that can be used to switch between debug/release (or even better development/staging/production) versions of a file. Think for example of a config.ts file that contains different parameters based on the environment.
I have done some research in the topic, but I was unable to find a conclusive answer:
the old and now retired documentation for the appbuilder platform mentions a (.debug. and .release.) naming convention for files. I don't think this work anymore.
other sources mention passing parameters during the call to tns build / tns run and then fetching them via webpack env variable... See here. This may work, but seems oddly convoluted
third option that gets mentioned is to use hooks to customize the build (or use a plugin that should do the same)
lastly, for some odd reason, the #nativescript/schematics seems to generate a default project that contains two files called environment.ts and environment.prod.ts. I suspect those only work for the web version of the project (read: ng serve) - I wasn't able to get the mobile compiler to recognize files that end with debug.ts, prod.ts or release.ts
While it may be possible that what I am trying to do isn't just supported (yet?), the general confusion an dissenting opinions on the matter make me think I may be missing something.. somewhere.
In case this IS somehow supported, I also wonder how it may integrate with the NativeScript Sidekick app that is often suggested as a tool to ease the build/run process of NativeScript applications (there is no way to specify additional parameters for the tns commands that the Sidekick automates, the only options available are switching between debug/release mode), but this is probably better to be left for another question.
Environment files are not yet supported, passing environment variables from build command could be the viable solution for now.
But of course, you may write your own schematics if you like immediate support for environment files.
I did not look into sharing environment files between web and mobile yet - I do like Manoj's suggestion regarding modifying the schematics, but I'll have to cross that bridge when I get there I guess. I might have an answer to your second question regarding Sidekick. The latest version does support "Webpack" build option which seems to pass the --bundle parameter to tns. The caveat is that this option seems to be more sensitive to typescript errors, even relatively benign ones, so you have to be careful and make sure to fix them all prior to building. In my case I had to lock the version of #types/jasmine in package.json to "2.8.6" in order to avoid some incompatibility between that and the version of typescript that Sidekick's cloud solution is using. Another hint is to check "Clean Build" after npm dependency changes are made. Good luck!
What's the difference between the Perl JSON modules below?
I have come across JSON::PP and JSON::XS. The documentation of JSON::PP says it is compatible with JSON::XS. What does that mean?
I am not sure what the difference between them are, let alone which of them to use. Can someone clarify?
Perl modules sometimes have different implementations. The ::PP suffix is for the Pure Perl implementation (i.e. for portability), the ::XS suffix is for the C-based implementation (i.e. for speed), and JSON is just the top-level module itself (i.e. the one you actually use).
As noted by #Quentin, this site has a good description of them. To quote:
JSON
JSON.pm is a wrapper around JSON::PP and JSON::XS - it also does a bunch of moderately crazy things for compatibility reasons, including extra shim code for very old perls [...]
JSON::PP
This is the standard pure perl implementation, and if you're not performance dependent, there's nothing wrong with using it directly [...]
JSON::XS
Ridiculously fast JSON implementation in C. Absolutely wonderful [...]
As you can see, just installing the top-level JSON module should do it for you. The part about compatibility just means that they both do the same thing, i.e. you should get the same output from both.
I installed the Perl JSON module a few years ago on a RHEL server I managed and it was a really straightforward process: just install (or build) the module from the CPAN site and you're done.
Installing should be a simple case of either using the OS package manager (if in GNU/Linux), using the cpan utility, or building from source. The OS package manager is recommended, as it helps keep things updated automatically.
To verify that it's installed, just try the following command from the terminal (assuming GNU/Linux):
$ perl -e 'use JSON;'
If it doesn't complain, then you should be good to go. If you get errors, then you should get ready to go in an adventure.
You can install JSON module, cpan install JSON
use JSON;
my $result = from_json($json);
if($result->{field})
{
# YOUR CODE
};
I've just started using IPython for interactive development and exploratory research, which I've found really exciting with all the cool features and possibilities. I am using the Anaconda package manager to manage dependencies, which includes IPython.
From what I've read, one goal of the IPython team is to eventually integrate Sage Math (CAS) into IPython, as a cell magic. Does anyone know if this is still under development? Or rather, if I wanted to use Sage now, is writing an extension the only way to do this [1]?
[1] https://github.com/ipython/ipython/wiki/Extensions-Index
Also, if I install additional packages for scientific development, not included in the Anaconda distribution, is that as easy as just pip or do I have to go through a Anaconda package build to handle dependencies and such? If I were only using IPython, I could understand just doing easy_install or pip as recommended in the docs, but I believe that overwrites existing dependencies within Anaconda. If I use pip, how does that affect Anaconda dependencies if I do not install in an Anaconda environment, which I take is the equivalent as virtualenv.py, and is this the way to also set up revision control (i.e. Mercurial)?
To clarify, I do not want to run IPython from within Sage, I want to run Sage, as a CAS, from within IPython. I'd rather go the Sage approach of integrating domain specific languages. Or in contrast, will IPython extensions replace Sage?
I'm a self taught programmer, not a professional software developer. As an engineer, I am used to Matlab, Mathematica, and commercial solutions, that allow me to abstract away the plumbing. I'm trying to wrap my mind around getting everything glued together, but it's like a mix of spaghetti soup and a dynamic link library, due to lack of knowledge. I'm probably using the wrong approach.
What I want is Anaconda/Enthought package management (IPython, pandas, etc..), custom rolled Sage through hooks/extensions or magics, extensions to packages not included in Anaconda (i.e. Matlab see [1] above), and revision control with Git and Mercurial. How would professional developers set this up on a Mac or Linux box?
Answering the first question:
Sage is a huge collection of mathematical software, including IPython. There's no way we'd integrate all of that into IPython.
Possibly what you've heard is that we're going to integrate Sage-style 'interacts' into IPython. That's where you have a slider to control the value of some input variable, and the output updates as you move it around, based on a calculation written in Python. That is still on our roadmap to add to IPython.
Another possibility is that you're thinking of SymPy, a Python-based CAS. SymPy works well within IPython, especially if you call sympy.init_printing() to get the fancy representations of expressions.
I wrote an IPython extension to load Sage customizations into the IPython notebook---in fact, that's how many of the customizations to IPython are done for the normal Sage interface. It basically turns the IPython notebook into an interface to Sage (e.g., preparsing is done, etc.).
You do need to run it from Sage's copy of IPython, though. Just start the IPython notebook:
sage -ipython notebook
and then load the sage extension in a cell:
%load_ext sage.misc.sage_extension
Pretty soon we'll upgrade to IPython 1.0 (I've made the changes necessary, and it needs to be reviewed). If you want to run IPython 1.0 already, email the sage-support mailing list and I'll post instructions.
To answer your other question, Sage includes many packages that are not available in Anaconda. Sage depends on these packages heavily for many features. I suppose there is a possibility getting Sage and its dependencies distributed with something like Anaconda, but no one is working on that as far as I know. There is some work on packaging Sage up for different linux distributions and replacing the package manager for Sage.
To clarify, I do not want to run Ipython from within Sage, I want to run Sage, as a CAS, from within Ipython. I'd rather go the Sage approach of integrating domain specific languages. Or in contrast, will Ipython extensions replace Sage?
If you want to run Sage within Ipython, the easiest thing to do is to use Sage's copy of Ipython:
$ sage -ipython
Python 2.7.5 (default, Aug 1 2013, 18:11:00)
Type "copyright", "credits" or "license" for more information.
IPython 0.13.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: from sage.all import *
In [2]: integrate(x^2,x)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-2-006357f5d9c0> in <module>()
----> 1 integrate(x^2,x)
NameError: name 'x' is not defined
In [3]: var('x')
Out[3]: x
In [4]: integrate(x^2,x)
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-4-006357f5d9c0> in <module>()
----> 1 integrate(x^2,x)
/Users/.../sage-5.11.rc0/local/lib/python2.7/site-packages/sage/structure/element.so in sage.structure.element.Element.__xor__ (sage/structure/element.c:6754)()
RuntimeError: Use ** for exponentiation, not '^', which means xor
in Python, and has the wrong precedence.
In [5]: integrate(x**2,x)
Out[5]: 1/3*x^3
Note that there are some differences between this and Sage - for instance, there is no preparsing of the syntax. Presumably if you had Sage someplace your own installation of Ipython can find it, you could do this there too (though there is no easy_install and the Python versions might not match correctly!).
Please observe that Sage, as it currently is and should be in the near future, is a huge package that runs in POSIX (Linux-like) OSes only. If you intend it to run in Windows you must use VirtualBox with Sage running under a VM.
If you want a simple CAS that is simple to install and runs everywhere that Python does, you should consider taking a look at Sympy. It doesn't interact with all the software that Sage does, but is simple to use, lightweight (no external libraries needed) and is already included in the base Anaconda distribution.
Besides, it can be easily used inside other Python programs, since it's just like any other module and doesn't make changes to the Python syntax.
Is there a place I can find Backus–Naur Form or BNF grammars for popular languages? Whenever I do a search I don't turn up much, but I figure they must be published somewhere. I'm most interested in seeing one for Objective-C and maybe MySQL.
you have to search on tools used to create grammars: "lex/yacc grammar", "antlr grammar" "railroad diagram"
http://www.antlr3.org/grammar/list.html
Here's some grammar files
objective-c
http://www.omnigroup.com/mailman/archive/macosx-dev/2001-March/022979.html
http://www.cilinder.be/docs/next/NeXTStep/3.3/nd/Concepts/ObjectiveC/B_Grammar/Grammar.htmld/index.html
https://github.com/pornel/objc2grammar
python
http://www.python.org/dev/summary/2006-04-16_2006-04-30/#the-grammar-file-and-syntaxerrors
javascript
http://tomcopeland.blogs.com/EcmaScript.html
http://www.ccs.neu.edu/home/dherman/javascript/
ruby
http://www.ruby-doc.org/docs/ruby-doc-bundle/Manual/man-1.4/yacc.html
FWIW, the MySQL grammar file (mysql-server/sql/sql_yacc.y) is open source and browseable at launchpad.net (though it's a bit slow and I got an error when I tried to pull up the specific file).
Also, a snapshot of the whole MySQL Server source is downloadable from dev.mysql.com.
There are some links from w:BNF#Language Grammars.
BNF Grammars for SQL-92, SQL-99 and SQL-2003
I also found a page that lists grammars for Objective C.
Objective-C grammar for Lex/Yacc Flex/Bison
Reference Manual for the Objective-C Language
IIRC, BNF grammars are just different enough from what yacc/bison want as input to be really annoying :) If you intend to feed these files into a parser generator, you may want to look for files in the appropriate format. I recall seeing such files for Java, JavaScript and C++ at one point. Probably as part of Eclipse, Firefox and GCC, respectively, but I can't remember for sure. I would assume you can find pretty much any parser input file by finding an open source project that uses that language.
I also searched this and i collected this repository
http://slps.github.io/zoo/