Creating a language on the Rubinius VM - vm-implementation

I'm looking to play around with the Rubinius VM to create a langauage, but just reading the documentation, I'm still quite lost on how to get started. Even looking at the projects, I still can't seem to figure out where the parsing and using the vm comes into place. Does anyone have any resources for this?

Hey I'm a contributor to the Fancy language that runs on rubinius. If you're interested in parsing take a look at boot/rbx-compiler there you'll find a Parser (implemented with KPEG) that basically constructs a tree of AST nodes, each of those nodes has a bytecode method that produces the rubinius vm instructions for everything to work. Fancy share a lot of semantics with ruby, so I guess starting with it would be easy if you're already familiar with ruby. You'll just need to checkout the examples/ dir to geet a feeling on the language and then the kpeg parser, ast nodes, loader, as you progress exploring the compiler. These days Fancy is bootstrapped (that means that the compiler has been written in fancy itself - at lib/compiler) but rbx-compiler is the first step in that process.
Hope exploring Fancy's source code can be of help to you.

In case you hadn't seen it, check out Evan's keynote from 2011 LA Ruby Conf. He shows how to build a simple language, which might be helpful.

Related

Extending embedded Python in C++ - Design to interact with C++ instances

There are several packages out there that help in automating the task of writing bindings between C\C++ and other languages.
In my case, I'd like to bind Python, some options for such packages are: SWIG, Boost.Python and Robin.
It seems that the straight forward process is to use these packages to create C\C++ linkable libraries (with mostly static functions) and have the higher language be extended using them.
However, my situation is that I already have a developed working system in C++ therefore plan to embed Python into it so that future development will be in Python.
It's not clear to me how, and if at all possible, to use these packages in helping to extend embedded Python in such a way that the Python code would be able to interact with the various Singleton instances already running in the system, and instantiate C++ classes and interact with them.
What I'm looking for is an insight regarding the design best fitted for this situation.
Boost.python lets you do a lot of those things right out of the box, especially if you use smart pointers. You can even inherit from C++ classes in Python, then pass instances of those back to your C++ code and have everything still work. My favorite resource on how to do various stuff is this (especially check out the "How To" section): http://wiki.python.org/moin/boost.python/ .
Boost.python is especially good if you're using smart pointers or intrusive pointers, as those translate transparently into PyObject reference counting. Also, it's very good at making factory functions look like Python constructors, which makes for very clean Python APIs.
If you're not using smart pointers, it's still possible to do all the things you want, but you have to mess with various return and lifetime policies, which can give you a headache.
To make it short: There is the modern alternative pybind11.
Long version: I also had to embed python. The C++ Python interface is small so I decided to use the C Api. That turned out to be a nightmare. Exposing classes lets you write tons of complicated boilerplate code. Boost::Python greatly avoids this by using readable interface definitions. However I found that boost lacks a sophisticated documentation and dor some things you still have to call the Python api. Further their build system seems to give people troubles. I cant tell since i use packages provided by the system. Finally I tried the boost python fork pybind11 and have to say that it is really convenient and fixes some shortcomings of boost like the necessity of the use of the Python Api, ability to use lambdas, the lack of an easy comprehensible documentation and automatic exception translation. Further it is header only and does not pull the huge boost dependency on deployment, so I can definitively recommend it.

JSP/.NET XML - simple client to display html

I am fairly proficient in PHP, but just starting out in ASP.Net and JSP/Java
I would like to learn JSP/ASP.NET XML to HTML transformation with some simple practical examples. Im not looking to learn how to edit XML, just displaying it, but im having trouble finding definitive examples/tutorials.
Ive spent quite a while studying JSP/ASP.NET but quickly find how vast they are and how many different ways there are to do this (quite frankly im a bit overwhelmed). I would be really grateful for advice before I embark upon this journey (and perhaps I will be saved from going in the completely wrong direction). If there are any tutorials or especially example apps you could point me towards this would really help (i like to do hands on learning)
For this I expect I need to do the following:
1) Set up a server for each technology (im using Tomcat and IIS at the moment - are these the best?)
2) Use some parameter based routing system (MVC?, but this is most surely overkill for me)
3) Parse the XML and create some variables/objects
4) Display the HTML (Use template libraries (JSTL? not sure for ASP.NET))
Any tutorials/example apps you could point me towards to help me through the above steps will be truly appreciated.
Thankyou
Ke
By the sounds of your skillset, carefully working through this developerworks tutorial on JSTL looks like a good place for you to start. It does cover the XML handling libs around part 4, and it'll also help you avoid the mistake of using scriptlets where JSTL would give cleaner, less error-prone and much more readable code.
You'll also most likely want IDE support, so that you get documentation, syntax checking and autocomplete. I personally use Eclipse (The EE download will have everything you need and more) but NetBeans might be the most straightforward to get your started.
Tomcat will be fine to get you started, but these IDEs tend to have build in web containers to save you time in deploying and testing.

Understanding run time code interpretation and execution

I'm creating a game in XNA and was thinking of creating my own scripting language (extremely simple mind you). I know there's better ways to go about this (and that I'm reinventing the wheel), but I want the learning experience more than to be productive and fast.
When confronted with code at run time, from what I understand, the usual approach is to parse into a machine code or byte code or something else that is actually executable and then execute that, right? But, for instance, when Chrome first came out they said their JavaScript engine was fast because it compiles the JavaScript into machine code. This implies other engines weren't compiling into machine code.
I'd prefer not compiling to a lower language, so are there any known modern techniques for parsing and executing code without compiling to low level? Perhaps something like parsing the code into some sort of tree, branching through the tree, and comparing each symbol and calling some function that handles that symbol? (Wild guessing and stabbing in the dark)
I personally wouldn't roll your own parser ( turning the input into tokens ) or lexer ( checking the input tokens for your language grammar ). Take a look at ANTLR for parsing/lexing - it's a great framework and has full source code if you want to dig into the guts of it.
For executing code that you've parsed, I'd look at running a simple virtual machine or even better look at llvm which is an open-source(ish) attempt to standardise the virtual machine byte code format and provide nice features like JITing ( turning your script compiled byte code into assembly ).
I wouldn't discourage you from the more advanced options that you machine such as native machine code execution but bear in mind that this is a very specialist area and gets real complex, real fast!
Earlz pointed out that my reply might seem to imply 'don't bother doing this yourself. Re-reading my post it does sound a bit that way. The reason I mentioned ANTLR and LLVM is they both have heaps of source code and tutorials so I feel this is a good reference source. Take it as a base and play
You can try this framework for building languages (it works well with XNA):
http://www.meta-alternative.net/mbase.html
There are some tutorials:
http://www.meta-alternative.net/calc.pdf
http://www.meta-alternative.net/pfront.pdf
Python is great as a scripting language. I would recommend you make a C# binding for its C API and use that. Embedding Python is easy. Your application can define functions, types/classes and variables inside modules which the Python interpreter can access. The application can also call functions in Python scripts and get a result back. These two features combined gives you a two-way communication scheme.
Basically, you get the Python syntax and semantics for free. What you would need to implement is the API your application exposes to Python. An example could be access to game logic functions and render functions. Python scripts would then define functions which calls these, and the host application would invoke the Python functions (with parameters) to get work done.
EDIT: Seems like IronPython can save you even more work. It's a C# implementation of CPython, and has its own embedding API: http://www.ironpython.net/

worth to learn groovy? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
the question im asking, is it worth to learn a new language like groovy? cause if i learn groovy, it feels like i code in groovy and not java. and how smart is that when i have to be good in java to code desktop applications too in the future. so if i use groovy a lot for web applications, i will just be worse and have to start over to be good in java when i code desktop applications right?
so why don´t I just stick with java and be good at ONE language instead of having to switch between 2 languages and their syntax. Cause it would be so confusing...
Groovy is a nice, scriptable and easier-to-use Java "knockoff" – and I don't mean that derogatively. while Java is a language to be compiled, deployed and (often) run on Enterprise servers where performance matters, Groovy is a language where you can quickly create a program to get something done. Often that something is fairly simple, so it's an hour's or a day's coding effort. Often the code is only run once and then thrown away. Because Java has more boilerplate and formalism in it, you can do this kind of program more quickly and hence more efficiently in Groovy.
However, just to give you some perspective, Groovy is a relative newcomer stomping on the turf of various other, better established scripting languages:
Perl is one of the grandfathers of scripting languages; rarely does a Unix server get installed without Perl on it, and Perl scripts are the lifeblood of many servers. However, Perl is a write-only language that looks like line noise to the uninitiated. There's more than one way to do everything, so styles diverge drastically. Perl coding tends to be a bit messy.
Python is a fresher, cleaner script language than Perl, and is these days preferred by many as a scripting language. It's fun to program in, it gets things done and because it's been around for a few years, lots of people know it. Python is found behind/inside a number of Linux system utilities.
Groovy leaves Perl and Python in the dust when (a) the environment already makes use of a JVM and/or there's a requirement to use existing Java code, including libraries. So far so good. Groovy is not blazingly fast, but faster than Python. Being dynamically typed, it's "fun" and "easy" to program in a way that Java's not.
But then came Scala. Scala is like Java on steroids. It is statically typed so it's not quite as "fun" to program as Groovy, but it has type inference so often you can leave off the types and the compiler can figure them out. Scala works really hard to make the most of types; it does generic types a lot more seamlessly than Java. It dispenses with a lot of Java's boilerplate, so Scala programs are typically about 30% shorter than similar Java programs. Scala runs on the JVM and interfaces pretty well with Java code. It also runs about as quickly as Java, which most of the other languages don't.
Finally in historic order, there's Clojure. Clojure is a Lisp derivative, so it has a programming style very different from languages you'd otherwise know, and it burns through a lot of parentheses! But Clojure runs on the JVM, is very compatible with all the rest of Java, and it's dynamically typed. You can use it as a scripting language or treat it like a compiled language... it's up to you. I find it fun to program in, and the fact that it's an almost pure functional language forces you to think in new ways about programming. It hurts your head at the beginning, but if you survive it's a very worthwhile exercise because you learn some techniques that will become more relevant (I think) in future programming.
In summary, it would probably do you good (put hair on your chest, if I may be so sexist) to learn one or more of these "alternative" / "scripting" languages. You may find them useful. Usually when there's something to be hacked up quickly in my project, I get the job because all my colleagues only know Java, and by the time they finish setting up their class framework I'm already done.
Quote:
so why don´t I just stick with java
and be good at ONE language instead of
having to switch between 2 languages
and their syntax.
This seems like a more general question about learning programming languages than learning a new language (Groovy) which runs on top of the Java Virual Machine.
Here's a question:
Suppose you are learning a foreign language because you want to be fluent in multiple languages so you can converse with many people. You're learning German right now, but you're getting good at it, but you also want to learn Spanish. Would you just suddenly forget German if you start to learn Spanish? If you are indeed worried that you will, what would you do?
If you were going to learn Groovy, but don't want to forget how to write Java, then why not continue to use both languages at the same time?
One of the things about being a programmer is going to be learning to adapt to new technologies as they come along. It's a good thing to be able to learn new languages, as it's going to be a skill that's going to be very useful in a field which is constantly changing.
Why don't you code your desktop apps in groovy too? Just because groovy is the choice of a web framework (grails) doesn't mean that you can't use it for desktop apps.
Indeed, it is great for desktop apps too. It's more a matter of dynamic or static languages...
In my opinion, it is quite good to have for each task the right language at hand. So go ahead and learn groovy - the result will be that you'll miss groovy features when you try to use java again ;-)
I would say in general in this field it's always good to be learning. I try constantly to learn new concepts to add to my toolbox, while getting better at the core things I'm interested in like Java. I recently purchased a book on learning Clojure - another functional language for the JVM.
The downside to learning something without using it every day is that some details don't stick in your head. That said, I'm glad I spent some time with Clojure; the important stuff stuck and I know I can quickly look up the details if and when I need to. You may want to take a similar approach to Groovy.
The Java platform is slowly starting to change direction to one where the JVM is targeted by multiple source languages (a trick .net has been showing off since day 1, but it's taking Java a while to catch up there). The Java7 classfile format is even adding a new instruction to make these dynamic languages work faster.
If you want to keep yourself current, then learning Groovy is a good way to do it, without abandoning all your investment in the Java platform.
Furthermore, Groovy (and Grails) is now maintained by SpringSource, so its popularity is only going to increase.
Going from java to groovy isn't a lot of work. No where near what would be needed to move to a less Javaish language like clojure.
I really like groovy for one-off apps and for scripting existing java code. I've used it to parse data from REST calls and feed the data to a JMS queue. I've used it to create scrambled test data for a partner from our production data. For stuff like that it is amazing.
If the goal is to learn a dynamic language to add to the toolbox, Python and Ruby are both good choices. They run on the JVM and have native versions. Both are well supported on a large number of platforms.
If the goal is to learn an alternative JVM language, groovy is an excellent choice. Both Scala and Clojure would also be good choices.
I used to stick to the "learn a new language every year rule" from The Pragmatic Programmer, but that was before I had kids. Now I learn a new building toy every six months.
First of all I'm this is a highly subjective question.
In my humble opinion it is worth learning a new language especially if it varies in paradigms (as is the case with groovy). I'm fairly young myself so for me learning a new language is not a much of hassle but the way I see it if you like the language, you estimate that coding in language X will be profitable you should learn it.
It won't hurt your resume.
It won't make your head hurt (much).
The only problem is, will you use it. You need to use a language to become good at it. If you are going to learn it now and never use it tomorrow it probably ain't worth learning it.
Learning something new does not take away something you already know. You may be a bit rusty when you get back into Java, but it'll come back real quick.
Also--
I'm not a Java guy, but I believe Groovy targets the JVM. If this is the case, then programming in Groovy will make you a better Java programmer, because you'll still be targeting the same framework as Java (the language) so you'll still continue to gain experience with the Java libraries. Knowing the available libraries is what really matters, not how well you know every minute detail of a particular language.
I find that by learning new languages, I always end up learning new ways to think about problems. Each language guides you into solving problems in the way most easily expressed by the language. Learning new languages only makes you stronger all around because you learn new ways to solve problems.
You might have to re-orient yourself with the libraries after a long time away from a language, but even then it's not a huge ordeal - just more frequent google searches, etc.
The benefits, however, are worth it. I recently did some functional programming for the first time and it really taught me a lot of different ways to think about certain situations. I find myself now using some of C#'s functional aspects and it makes my code a lot cleaner in some cases. The bottom line is; if your going to do this for a living you are going to want to learn more than one language, have you ever met a mechanic that only knew one make and model of car?
It's always good to learn a new language to be a better programmer. Groovy is a natural choice for java programmer - easy to learn and you can still use your all java knowlege.
Groovy is a dynamic language, after try to learn any functional language (like Scala). With this experience you will see java from different perspecitve. Some task that was painful in Java will be trivial in Groovy/Scala.
you can program desktop aplication with Griffon whose language of choice is Groovy, give it a try
If you are looking for online help, check this websites:
for Groovy
for Grails

Adding a language to the AVM2

I'm interested in making a language to run on the AVM2 and I'm looking for advice on where to start. I do realize that this is by no means a trivial task, but I would like to give it a try and at the very least learn more about implementing a language along the way.
I have messed around with ANTLR and have been reading up on syntax issues for language development. What I'm looking for is advice on a path to take or useful references/books.
For instance I would like to generate (script/manually) some very simple AVM2 bytecode and get that to run on the VM as a start.
Thanks
If you are not interested in Haxe, you will basically need to write your own compiler that compiles objects down to ABC (Actionscript Byte Code). The AVM2 Overview document available from Adobe on ABC and the AVM2 which should help you get started. It's a fairly thorough document but stay alert for a few typo's in the bytecode instructions.
You will also need to wrap the bytecode in a doABC tag as part of a SWF container. You can get more information from the SWF File Format documentation.
If you'd like a headstart on writing the data structures (optimised int formats, etc), feel free to checkout the code at asmock, a dynamic mocking project I've been working on. The SWF/ByteCode generation stuff is a bit messy but there are IDataOutput wrappers (SWF, ByteCode) that might come in handy.
Project Alchemy by Adobe can be a good reference
http://labs.adobe.com/technologies/alchemy/
How did it go?
I'm also interested in doing a Java to AVM2 compiler...
Do you have any published code?
Take a look at Haxe: it is an open source language that can target different platforms, including the AVM. You can dig into the SWF compiler source code to get some inspiration.