I am currently working on a design for a collection of subsystems, and I would like to be able to offer the API's exposed by a given subsystem for use by other subsystems.
In the past, I have used SWIG to expose C api's to a variety of other languages. This has worked well for me, but ultimately the API is defined in C. So basically one side of the API is language agnostic, and the other isn't.
What I would really like is to have something similar to SWIG that could generate the interface between 2 arbitrary languages based on some description of the API.
I don't want to use web services.
For example, I would like to call a 'function' from java, and implement the 'function' in Python. I'd like to be able to generate the language interop using a code generator.
Is there anything that exists which can do this today? At least for simple 'function' calls - ignoring the more complex cases like callbacks and situations where you need to maintain references outside of the 'function' call itself.
I found this after following gooli's link to Protocol Buffers .
This looks so compelling I'm going to propose it as an answer to my own question.
http://incubator.apache.org/thrift/
Thrift is a software framework for scalable cross-language services development. It combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, Smalltalk, and OCaml.
From the whitepaper:
...we were presented with the challenge of building a transparent,
high-performance bridge across many
programming languages. We found that
most available solutions were either
too limited, did not offer sufficient
datatype freedom, or suffered from
subpar performance
Also from the whitepaper:
A. Similar Systems The following are software systems similar to Thrift.
Each is (very!) briefly described:
SOAP. XML-based. Designed for web services via HTTP, excessive XML
parsing overhead.
CORBA. Relatively comprehensive, debatably overdesigned and
heavyweight. Comparably cumbersome
software installation.
COM. Embraced mainly in Windows client softare. Not an entirely open
solution.
Pillar. Lightweight and high-performance, but missing
versioning and abstraction.
Protocol Buffers. Closed-source, owned by Google. Described in Sawzall
paper.
The last part about Protocol Buffer's is out-of-date - its been open sourced
COM (and to a lesser extent Firefox's XPCOM) might be what you are looking for. In COM, you define the API using a language called IDL which can then be compiled into different languages. When using the same language the calls usually degenerate into efficient function calls.
However, COM is a very complex and dying piece of technology that I would never use for a new project.
Related
What is an embedded scripting language? What are some advantages it would have over the domain specific language?
"A number of languages have been designed for the purpose of replacing application-specific scripting languages by being embeddable in application programs." -Scripting language (wiki)
Javascript.NET is one such example. It can be used in place of c# for example. However, what constitutes the difference between a scripting language, and an embedded scripting language? Moreover, why would javascript be more desirable than C# - or any other domain specific language?
An embedded scripting language would be a scripting language (or more specifically an interpreter for such a language) that can be embedded into applications. Thus, the interpreter has an API that can be used to integrate it into the application, allowing scripts to control all or parts of the application.
the advantage is that application developers only have to provide the interfacing with the language; they don't have to implement the actual language. This allows more complex and feature-rich languages to be used, since there's no development time (from the application developer point of view).
For users, it means they know the pros and cons, the quirks and benefits of the scripting language, provided we are talking about a commonly used one.
For your core business, you probably should not use an embedded language, but there are other upsides of using an embedded language. Most importantly, in my experience, it is beneficial to have an easy-to-use language on which other users can build against your system. While theoretically it is possible to build against your system with proper definition of interfaces, it is a lot more user-friendly to let inexperienced people play with JavaScript and Lua, instead of having to set-up an entire environment to deploy in .NET.
Of course, it is easy to hot-swap embedded languages and it can speed up your testing considerably; it is also possible to see your changes immediately without reloading your application, because you just reload a small virtual machine for that embedded language.
I personally employed this to allow people to write assisting scripts against a game.
Just looking at Haskell and web frameworks and wondering if it would make sense to use Haskell's great threading/event functionality to power a platform for writing HTML5 and REST apps that expose an HTTP API for data and a WebSocket (with maybe SockJS support for appropriate fallback) API for events? It doesn't seem like the "big" web frameworks support WebSockets as a first-class citizen, though they seem to have a lot of other things going for them.
My concern is making use of available cores, which Haskell can do well, but also providing easy user integration on the server side for validation and server-side logic (maybe by embedding Lua or similar?). If one wrote this on the JVM, one could make use of multiple server-side language support and lots of libraries for this sort of thing.
I'm sure people are doing things like this in a one-off solution for their own applications but I'm thinking along the lines of a PaaS-type approach where one can write HTML5 apps with data (including proper synchronization for offline use) and eventing "for free" as a fundamental part of the platform. Most logic would reside in the browser but some could be run on the server with the appropriate hooks and a reasonable embeddability (JavaScript seems out of the question and not sure about embedding interpreters in Haskell as I'm only dangerously familiar with Haskell in general).
Part of the problem I've had with Haskell so far is that I'm not a Math guy. I didn't study CS in college and I'm a creative-type thinker. So a lot of the tutorials and documentation get me pretty lost, especially when dealing with the mathematical stuff.
Has anyone trod this path already? Am I late to the party? :)
Gregory Collins gave a tutorial at CUFP last year about using Snap to build an interactive chat website using long polling (not websockets). The source code is here.
In the websockets department, Jasper Van der Jeugt wrote a Haskell websockets library. It is available on hackage and comes with websockets-snap, which provides Snap framework support. There's also wai-websockets which provides integration with Warp.
I believe all of the major frameworks have some level of websockets support, so they should all be a fair choice based on your requirements. For Yesod, there's an example of creating a chat system (using eventsource, not websockets) available in the book:
http://www.yesodweb.com/book/wiki-chat-example
Is there any language-independent tutorial or book for developing web applications?
For example implementation of login procedures, photo-gallery making etc, not focused on any particular language?
I'd suggest a book like Web Application Architecture: Principles, Protocols and Practices. It provides a high level overview of the pieces of developing a web application, with some real examples when they are called for.
I'd like to suggest - Coding Horror: Recommended Reading for Developers
Patterns of Enterprise Application Architecture by Martin Fowler is a good start.
The example code is in Java and .Net but the patterns transcend any specific programming language. It deals with subjects such as how to implement server side sessions, patterns used to create a Model-View-Controller framework, patterns used to create an object relational mapping framework, and a little bit on how to string it all together.
A few other paradigms have entered into the fray since it was written, which are not covered such as dependency injection. There is no coverage of NoSql databases and when to use them. It also lacks coverage of more retro topics such as CGI, and older frameworks/languages that used this standard (ie webapps written in Small talk). The book also predates the widespread use of "convention over configuration" frameworks such as Rails. Transactions are dealt with adequately but the coverage is not complete, referring the reader to Principles of Transaction Processing by Bernstein and Newcomer, for in depth information.
It might be worthwhile to complement reading PEAA with a good understanding of Unix sockets, and how this ties up with the threading and shared memory facilities offered by the language you are programming your web application in. The concurrency features of the run-time environment your application is deployed in will influence how you process requests and implement server-side sessions. If you need to remind yourself about sockets read chapter 12 of "The Illustrated Network" by Goralski. If you really want to geek out read the chapters on networking and threading in "Computer Systems: A Programmer's Perspective" by Bryant and O'Hallaron, though this might be overkill. Just know the concurrency capabilities of your chosen programming language and how it talks to the network layer.
A good understanding of the HTTP protocol is essential. This will lay a solid foundation for learning other important standards based on HTTP such as REST and AJAX. For in depth information on HTTP there is "HTTP: The Definitive Guide" by Gourley and Totty. There is also W3C. For an introduction to REST have a look at
http://www.infoq.com/articles/rest-introduction
For a broad brush overview of HTTP and sockets have a look at Wikipedia.
Concerning usability I would recommend the following two books from the Pragmatic bookshelf:
http://pragprog.com/titles/bhgwad/web-design-for-developers
http://pragprog.com/titles/jsaccess/design-accessible-web-sites
Finally concerning security a good place to start might be "24 Deadly Sins of Software Security" by Howard, LeBlanc and Viega. Part 1 deals specifically with web applications and Part 4 deals with network security in general. Part 3 on cryptography is also good, if you are generating "remember-me" tokens stored in a cookie, so that a user does not have to log in each time they use your app.
I am not aware of any one "cookbook" that will provide a solution for every problem you are likely to come across developing a web app but if you read all of the above you will be fairly well positioned.
What is meant by framework, programming language and scripting language?
I think Daniel Pryden's points are excellent - I voted him up. I'd just like to add a couple of minor additions.
Programming languages, like C and C++, used to have a compiling and linking step that rendered the source code into a machine-language form that was run by the operating system.
Scripting languages, like the Unix Bourne, Korn, and C shell, were not compiled or linked like C and C++. (Thanks to Daniel Pryden's correction and citation of the Unix scripting languages.)
Since virtual machines have become so common in languages like Java, Python, and C#, the distinction between scripting and programming has been blurred.
As for the distinction between libraries and frameworks: your code links in libraries and calls them. This is different from a framework, because your code is plugged into the framework. It's known as the Hollywood principle: "Don't call us, we'll call you."
The distinction between programming languages and scripting languages is ill-defined. In my opinion, we should dispense with the term "scripting language" and just call them all programming languages.
A framework, on the other hand, is a collection of code that uses an inversion-of-control mechanism to help you structure your code. Frameworks are similar to libraries in that they provide building blocks you can use to build a bigger system.
Frameworks are libraries or templates of pre-written stuff you can re-use. They often come with a pre-imposed structure and philosophy of how to approach a domain of problems.
Programming language is a superset of scripting language and includes anything you yourself write that makes a computer do something.
Scripting languages are traditionally interpreted meaning they are useful for smaller more light-weight purposes, but that definition is getting vaguer since JavaScript - a 'scripting language', for example, is not interpreted so much these days.
Framework is a combination of class libraries and runtime environment to execute code by independent on operating system. It is placed on operating system, mask functionalities of OS.
Scripting language is a language, it has script nothing but a support to another language its not used individual, like JavaScript, asp.net server side scripting, these are all give support and prompting to other languages, like JavaScript give prompting to html.
Programming language is instruction to computer and then computer execute that one
Framework is a combination of class libraries and runtime environment to execute code by independent on operating system. It is placed on operating system, mask functionalities of OS.
Scripting language is a language, it has script nothing but a support to another language its not used individual, like JavaScript, asp.net server side scripting, these are all give support and prompting to other languages, like JavaScript give prompting to html.
Programming language is instruction to computer and then computer execute that one
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I need some resources that talk about how to design your software to be extensible, i.e. so that other people can write add-ons/plug-ins that adds functionality to it.
What do you recommend? Any books out there that discuss the subject?
I would prefer something that's short and to the point; a bit of theory and a bunch of concrete examples.
I'm not targeting a specific language, I want to be able to understand the core idea so that I can implement it in any language.
And for the same reason, I prefer not to do it using a framework that someone else built (unless the framework is not very high-level, i.e. doesn't hide too much), at the moment I only want to educate myself on the subject and experiment with various ways to implement it. Plus, a framework usually assumes user's knowledge about the subject.
UPDATE
I'm not asking about OOP or allowing my classes to be inherited. I'm talking about designing an application that will be deployed on a system, such that it can be extended by third-party add-ons AFTER its been deployed.
For example, Notepad++ has a plug-in architecture where you can place a .dll file in the plugins folder, and it adds functionality to the application that wasn't there, such as color-picking, or snippet insertion, or many other things (a wide range of functionality).
IF we're talking .NET, try Scripting .NET applications with VBScript over on CodeProject. Lots of concrete examples there.
Below are sites implementing various application extension techniques
ClearScript - Makes V8, VBScript and JScript available to .NET apps
CS-Script - The C# Script Engine
Plugin Architecture using C#
Opinio plugin architecture
Notes on the Eclipse Plug-in Architecture
Plug-in Architecture Framework for Beginners
Gecko plugin architecture
Fungimol plugin architecture
OSGI is a good practical example of a technical framework allowing to do what you are after.
The theory is here.
The (free!) book is there.
Extensibility and the ability to write plugin must deal with service lifecycle
adding / removing service/plugin on the spot
managing dependencies between services
managing states of services (declared, installed, started, stopped,...)
What is OSGI for ?
One of the main functions of a module is as a unit of deployment… something that we can either build or download and install to extend the functionality of our application.
You will find a good introduction here, on the central notion of service (which is related to your question, and which explain some problems around services, key component for extensibility).
Extract:
Why are services then so important if so many applications can be built without them? Well, services are the best known way to decouple software components from each other.
One of the most important aspects of services is that they significantly minimize class loading problems because they work with instances of objects, not with class names. Instances that are created by the provider, not the consumer. The reduction of the complexity is quite surprising
Not only do services minimize configuration, they also significantly reduce the number of shared packages.
Implement SOLID principles in your application.
1. Single responsibility principle: A class should have only a single responsibility (i.e. only one potential change in the software's specification should be able to affect the specification of the class
2.Open/closed principle: Software entities … should be open for extension, but closed for modification
3. Liskov substitution principle: Objects in a program should be replaceable with instances of their subtypes without altering the correctness of that program
4. Interface segregation principle: Many client-specific interfaces are better than one general-purpose interface
5. Dependency inversion principle: One should Depend upon Abstractions. Do not depend upon concretions
Stackoverflow questions:
Example of Single Responsibility Principle
Is the Open/Closed Principle a good idea?
What is the Liskov Substitution Principle?
Interface Segregation Principle- Program to an interface
What is the Dependency Inversion Principle and why is it important?
You try to reach two competing goals:
The components of your software must expose a lot of themselves, so they can be reused
The components of your software must expose very little of themselves, so they can be reused
Explanation: To encourage code reuse, you should be able to extend existing classes and call their methods. This isn't possible when the methods are declared "private" and the classes are "final" (and can't be extended). So to meet this goal, everything should be public and accessible. No private data or methods.
When you release the second version of your software, you will find that many of the ideas of version 1 were plain wrong. You need to change many interfaces or your code, method names, delete methods, break the API. If you do this, many people will turn away. So in order to be able to evolve your software, the components must not expose anything that is not absolutely necessary - at the cost of code reuse.
Example: I wanted to observe the position of the cursor (caret) in an SWT StyledText. The caret is not meant to be extended. If you do it, you'll find that the code contains checks like "is this class in the package org.eclipse.swt" and a lot of methods are private and final and whatnot. I had to copy about 28 classes out of SWT into my project just to implement this feature because everything is locked down.
SWT is a nice framework to use and hell to extend.
Of course there is the famous Open Closed Principle - http://en.wikipedia.org/wiki/Open/closed_principle
Well it depends on the language.
In C/C++ I'm pretty sure there is a loadlibrary function that allows you to open a library at runtime and invoke it's exported functions. This is typically how it's done in C/C++.
In .NET, there is Reflection, which is offers similar (but more broad) to loadlibrary. There is also entire libraries built on Reflection like Managed Extension Framework, or Mono.Addins that does most of the heavy lifting for you already.
In Java, there is also Reflection. And there is the JPF (Java Plugin Framework) which is used in stuff like Eclipse IIRC.
Depending on what language you use I could recommend some tutorial/books. I hope this was helpful.
Plugin architecture is becoming very popular for its extensibility and thus flexibility.
For c++, Apache httpd server is actually plugin based, but a concept of module is used instead. Most of apache features are implemented as modules, like cache, rewrite, load balancing, and even threading model. It is a very modular software I ever saw.
And for java, Eclipse is definitely plugin based. The core of Eclipse is an OSGI module system which manage bundles, another concept for plugin. Bundle can provide extension points on which we can build modules with less efforts. The most intricate thing in OSGI is its dynamic characteristic, which means bundles can be installed or uninstalled at runtime. No stop-the-world syndrome any more!
Since I dont have enough rep points to leave a comment, I am posting this as an answer. SharpDevelop is an IDE for developing applications in C#/VB.NET/Boo. It has a pretty impressive architecture that allows itself to be extended in a number of ways - right from new menu items to development support for whole new languages.
It uses a bit of XML configuration to act as a glue layer between a core of the IDE and the plugin implementation. It handles locating, loading and versioning of plugins out of the box. Deploying new plugins is matter of simply copying in the new xml configuration file and the required assemblies (DLLs) and restarting the application. You can read more on this in the book "Dissecting a csharp application" by the original author(s) - Christian Holm, Mike Krüger, Bernhard Spuida of the application from here. The book doesnt seem to be available on that site, but i found a copy that might still be around here
Also found a related question here
Checkout "CAB" - Microsoft's Composition Application Building blocks Framework. I think they've got a "web version" of that too...
I have just started to develop a smart client application. These are two options I am considering.
Using Microsoft's System.AddIn namespace. Looks very promising, however it may be a little complex for our end solution.
Or the Smart Client - Composite UI Application Block from Microsoft
Recently, i have looked at taking components both the Composite UI Application Block and the System.AddIn namespace to build my own. Since source code is available for the CAB it is easy to extend. I think our end solution will be a light weight version of the CAB, definatly using the Unity Application Block
If you work with .Net, our research yielded two approaches: scripting and composition.
Scripting
You extend the functionality of what your classes can do by orchestrating them using scripts. That means exposing what is compiled in your favorite .Net language in a dynamic language.
Some options we found worth exploring:
IronPython
IronRuby
JavaScript: Jint, Jurassic and JavaScript .Net are good starting points.
Script.Net -> this one was the first one to call our attention.
Composition
If you start a project with .Net 4 or above, you must take a good look at the Managed Extensibility Framework (MEF). It allows you to extend the functionality of your apps in a plugin way.
The Managed Extensibility Framework (MEF) is a composition layer for
.NET that improves the flexibility, maintainability and testability of
large applications. MEF can be used for third-party plugin
extensibility, or it can bring the benefits of a loosely-coupled
plugin-like architecture to regular applications.
Managed Add-in Framework is also a good read.
MSDN: http://msdn.microsoft.com/en-us/library/dd460648.aspx
Codeplex: http://mef.codeplex.com/
Rather than re-inventing the wheel, use the frameworks in hand. Eclipse and Netbeans both support plugin based extensions. You have to work in Java though.