Native support for differential programming has been added to Swift for the Swift for Tensorflow project. Julia has similar with Zygote.
What exactly is differentiable programming?
what does it enable? Wikipedia says
the programs can be differentiated throughout
but what does that mean?
how would one use it (e.g. a simple example)?
and how does it relate to automatic differentiation (the two seem conflated a lot of the time)?
I like to think about this question in terms of user-facing features (differentiable programming) vs implementation details (automatic differentiation).
From a user's perspective:
"Differentiable programming" is APIs for differentiation. An example is a def gradient(f) higher-order function for computing the gradient of f. These APIs may be first-class language features, or implemented in and provided by libraries.
"Automatic differentiation" is an implementation detail for automatically computing derivative functions. There are many techniques (e.g. source code transformation, operator overloading) and multiple modes (e.g. forward-mode, reverse-mode).
Explained in code:
def f(x):
return x * x * x
∇f = gradient(f)
print(∇f(4)) # 48.0
# Using the `gradient` API:
# ▶ differentiable programming.
# How `gradient` works to compute the gradient of `f`:
# ▶ automatic differentiation.
I never heard the term "differentiable programming" before reading your question, but having used the concepts noted in your references, both from the side of creating code to solve a derivative with Symbolic differentiation and with Automatic differentiation and having written interpreters and compilers, to me this just means that they have made the ability to calculate the numeric value of the derivative of a function easier. I don't know if they made it a First-class citizen, but the new way doesn't require the use of a function/method call; it is done with syntax and the compiler/interpreter hides the translation into calls.
If you look at the Zygote example it clearly shows the use of prime notation
julia> f(10), f'(10)
Most seasoned programmers would guess what I just noted because there was not a research paper explaining it. In other words it is just that obvious.
Another way to think about it is that if you have ever tried to calculate a derivative in a programming language you know how hard it can be at times and then ask yourself why don't they (the language designers and programmers) just add it into the language. In these cases they did.
What surprises me is how long it to took before derivatives became available via syntax instead of calls, but if you have ever worked with scientific code or coded neural networks at at that level then you will understand why this is a concept that is being touted as something of value.
Also I would not view this as another programming paradigm, but I am sure it will be added to the list.
How does it relate to automatic differentiation (the two seem conflated a lot of the time)?
In both cases that you referenced, they use automatic differentiation to calculate the derivative instead of using symbolic differentiation. I do not view differentiable programming and automatic differentiation as being two distinct sets, but instead that differentiable programming has a means of being implemented and the way they chose was to use automatic differentiation, they could have chose symbolic differentiation or some other means.
It seems you are trying to read more into what differential programming is than it really is. It is not a new way of programming, but just a nice feature added for doing derivatives.
Perhaps if they named it differentiable syntax it might have been more clear. The use of the word programming gives it more panache than I think it deserves.
EDIT
After skimming Swift Differentiable Programming Mega-Proposal and trying to compare that with the Julia example using Zygote, I would have to modify the answer into parts that talk about Zygote and then switch gears to talk about Swift. They each took a different path, but the commonality and bottom line is that the languages know something about differentiation which makes the job of coding them easier and hopefully produces less errors.
About the Wikipedia quote that
the programs can be differentiated throughout
At first reading it seems nonsense or at least lacks enough detail to understand it in context which is why I am sure you asked.
In having many years of digging into what others are trying to communicate, one learns that unless the source has been peer reviewed to take it with a grain of salt, and unless it is absolutely necessary to understand, then just ignore it. In this case if you ignore the sentence most of what your reference makes sense. However I take it that you want an answer, so let's try and figure out what it means.
The key word that has me perplexed is throughout, but since you note the statement came from Wikipedia and in Wikipedia they give three references for the statement, a search of the word throughout appears only in one
∂P: A Differentiable Programming System to Bridge Machine Learning and Scientific Computing
Thus, since our ∂P system does not require primitives to handle new
types, this means that almost all functions and types defined
throughout the language are automatically supported by Zygote, and
users can easily accelerate specific functions as they deem necessary.
So my take on this is that by going back to the source, e.g. the paper, you can better understand how that percolated up into Wikipedia, but it seems that the meaning was lost along the way.
In this case if you really want to know the meaning of that statement you should ask on the Wikipedia talk page and ask the author of the statement directly.
Also note that the paper referenced is not peer reviewed, so the statements in there may not have any meaning amongst peers at present. As I said, I would just ignore it and get on with writing wonderful code.
You can guess its definition by application of differentiability.
It's been used for optimization i.e. to calculate minimum value or maximum value
Many of these problems can be solved by finding the appropriate function and then using techniques to find the maximum or the minimum value required.
(This is a CS-theory type of question; I hope that's acceptable.)
The "Lisp-1 vs Lisp-2" debate is about whether the namespace of functions should be distinct from the namespace of all other variables, and it's relevant in dynamically typed languages that allow the programmer to pass around functions as values. Lisp-1 languages (such as Scheme) have one namespace, so you can't have both a function named f and also an integer named f (one would shadow the other, just like two integers named f). Lisp-2 languages (such as Common Lisp) have two namespaces, so you can have both f variables, but you have to specify which one you mean with special syntax (#'f is the function and f is the integer).
It seems to me that the main technical problem, the need to disambiguate the function from the integer, is not an issue if the language is also statically typed (unlike most Lisps). For instance, if a sort function requires a list and a less-than function as an explicit signature,
def sort[X](list: List[X], lessThan: Function[X, Boolean]) // Scala syntax; had to pick something
then it wouldn't matter if the functions and everything else are in the same namespace or not. sort(mylist, myless) would only pass a type check if myless is a function--- no special syntax needed. Some people argue that one namespace is more aesthetically pleasing than two namespaces, but I'd like to focus on technical issues.
Is there anything that two namespaces would make more difficult or more prone to error (or conversely for one namespace), assuming that the language in question is statically typed?
(I'm thinking about this in the context of a domain specific language that I'm working on, and I want to make sure that I don't run into problems down the road. It would be easier to implement with two namespaces (Lisp-2), and since it's statically typed, there's no need for the equivalent of #'f. I asked the question in general because I want to hear general points and perhaps become aware of questions that I don't yet know to ask.)
One very common objection to multiple namespaces is that it complicates the formal semantics to have an arbitrary number of namespaces. One namespace makes things simple. The next simplest, so I'm told (I don't write these things), is an infinite number of namespaces--something I tried to do once and only got halfway through (see here if curious, though I think it's not what you're asking for in this case). It's when you limit it to a finite number of namespaces that a formal semantics gets messy, or so I'm told. And certainly it makes any kind of compiler or interpreter a bit more complex, too. This objection straddles between aesthetic and technical in that it's not an objection based on technical difficulty per se, as no super-human intelligence is required to do multiple namespaces, just a lot of extra manual labor, but rather an objection that doing a more complex semantics means more code, more special cases, more chances for error, etc. Personally, I'm not swayed by such arguments, I merely point them out since you ask. I think you'll find such arguments aren't fatal and that it's fine to proceed on what you're doing and see where it leads. I care much more about programmer/end-user experiences than implementor difficulty. But I mention this argument because other people I respect seem to think this is a big deal and I believe it's the correct answer to your question about difficulty and error-prone-ness.
Note, by the way, that when Gabriel and I wrote Technical Issues of Separation in Function Cells and Value Cells, I needed words to help me avoid saying "Scheme-like" and "Lisp-like" because people had unrelated reasons for preferring Scheme that had nothing to do with namespacing. Using the terms "Lisp1" and "Lisp2" allowed me keep the paper from becoming a Scheme vs. Lisp debate, and to focus readers narrowly on the matter of namespace. In the end, ANSI Common Lisp ended up with at least 4 namespaces (functions, values, go tags, block tags) or maybe 5 if you count types (which are sort of like a namespace in some ways but not others), but in any case not 2. So strictly it would be a Lisp4 or Lisp5. But either way it still horrifies those who prefer a single namespace because any arbitrary finite number bigger than one tends to be unacceptable to them.
My personal recommendation is that you should design the language according to your personal sense of what's right. For some languages, one namespace is fine. For others not. Just don't do it because someone tells you that either way has to be the right way--it's really legitimately your choice to make.
Some argue that one namespace is conceptually simpler, but I think that depends on the notion of simplicity. Some say smaller is simpler (notationally or implementationally). I claim that what's conceptually simplest is what's most isomorphic to how your brain thinks about things, and that your brain may not always be thinking "small" for "simple". I'd cite as at least one example that every human language in existence has multiple definitions for the same word, almost all resolved by context (of which your type inferencing might be one example of contextual information), suggesting that wetware is designed to disambiguate such things and that your brain is wasting some of its natural capacity if you don't let it care about such things. Maybe indeed this contextual inferencing increases the complexity of your language implementation, but languages are implemented only once, and are used many times, so there's every reason to optimize the end user experience, not the implementor experience. It's an investment that pays off later.
Insist on thinking about things how you want. Worry more about making your language have a good feel and about whether what you want to do is implementable at all, even if it does require work and extra checking of the code. No language design decision is the right one for every language--languages are ecologies and what matters is that language elements work well with one another, not that language elements match other languages. Indeed, if languages were going to be all the same, it would be pointless to have multiple languages.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
When all you have is a pair of bolt cutters and a bottle of vodka, everything looks like the lock on the door of Wolf Blitzer's boathouse. (Replace that with a hammer and a nail if you don't read xkcd)
I currently program Clojure, Python, Java and PHP, so I am familiar with the C and LISP syntax as well as the whitespace thing. I know imperative, functional, immutable, OOP and a couple type systems and other things. Now I want more!
What are languages that take a different approach and would be useful for either practical tool choosing or theoretical understanding?
I don't feel like learning another functional language(Haskell) or another imperative OOP language(Ruby), nor do I want to practice impractical fun languages like Brainfuck.
One very interesting thing I found myself are monoiconic stack based languages like Factor.
Only when I feel I understand most concepts and have answers to all my questions, I want to start thinking about my own toy language to contain all my personal preferences.
Matters of practicality are highly subjective, so I will simply say that learning different language paradigms will only serve to make you a better programmer. What is more practical than that?
Functional, Haskell - I know you said that you didn't want to, but you should really really reconsider. You've gotten some functional exposure with Clojure and even Python, but you've not experienced it to its fullest without Haskell. If you're really against Haskell then good compromises are either ML or OCaml.
Declarative, Datalog - Many people would recommend Prolog in this slot, but I think Datalog is a cleaner example of a declarative language.
Array, J - I've only just discovered J, but I find it to be a stunning language. It will twist your mind into a pretzel. You will thank J for that.
Stack, Factor/Forth - Factor is very powerful and I plan to dig into it ASAP. Forth is the grand-daddy of the Stack languages, and as an added bonus it's simple to implement yourself. There is something to be said about learning through implementation.
Dataflow, Oz - I think the influence of Oz is on the upswing and will only continue to grow in the future.
Prototype-based, JavaScript / Io / Self - Self is the grand-daddy and highly influential on every prototype-based language. This is not the same as class-based OOP and shouldn't be treated as such. Many people come to a prototype language and create an ad-hoc class system, but if your goal is to expand your mind, then I think that is a mistake. Use the language to its full capacity. Read Organizing Programs without Classes for ideas.
Expert System, CLIPS - I always recommend this. If you know Prolog then you will likely have the upper-hand in getting up to speed, but it's a very different language.
Frink - Frink is a general purpose language, but it's famous for its system of unit conversions. I find this language to be very inspiring in its unrelenting drive to be the best at what it does. Plus... it's really fun!
Functional+Optional Types, Qi - You say you've experience with some type systems, but do you have experience with "skinnable* type systems? No one has... but they should. Qi is like Lisp in many ways, but its type system will blow your mind.
Actors+Fault-tolerance, Erlang - Erlang's process model gets a lot of the buzz, but its fault-tolerance and hot-code-swapping mechanisms are game-changing. You will not learn much about FP that you wouldn't learn with Clojure, but its FT features will make you wonder why more languages can't seem to get this right.
Enjoy!
What about Prolog (for unification/backtracking etc), Smalltalk (for "everything's a message"), Forth (reverse polish, threaded interpreters etc), Scheme (continuations)?
Not a language, but the Art of the Metaobject Protocol is mind-bending stuff
I second Haskell. Don't think "I know a Lisp, so I know functional programming". Ever heard of type classes? Algebraic data types? Monads? "Modern" (more or less - at least not 50 years old ;) ) functional languages, especially Haskell, have explored a plethora of very powerful useful new concepts. Type classes add ad-hoc polymorphism, but type inference (yet another thing the languages you already know don't have) works like a charm. Algebraic data types are simply awesome, especially for modelling trees-like data structures, but work fine for enums or simple records, too. And monads... well, let's just say people use them to make exceptions, I/O, parsers, list comprehensions and much more - in purely functional ways!
Also, the whole topic is deep enough to keep one busy for years ;)
I currently program Clojure, Python, Java and PHP [...] What are languages that take a different approach and would be useful for either practical tool choosing or theoretical understanding?
C
There's a lot of C code lying around---it's definitely practical. If you learn C++ too, there's a big lot of more code around (and the leap is short once you know C and Java).
It also gives you (or forces you to have) a great understanding of some theoretical issues; for instance, each running program lives in a 4 GB byte array, in some sense. Pointers in C are really just indices into this array---they're just a different kind of integer. No different in Java, Python, PHP, except hidden beneath a surface layer.
Also, you can write object-oriented code in C, you just have to be a bit manual about vtables and such. Simon Tatham's Portable Puzzle Collection is a great example of fairly accessible object-oriented C code; it's also fairly well designed and well worth a read to a beginner/intermediate C programmer. This is what happens in Haskell too---type classes are in some sense "just another vtable".
Another great thing about C: engaging in Q&A with skilled C programmers will get you a lot of answers that explain C in terms of lower-level constructs, which builds your closer-to-the-iron knowledge base.
I may be missing OP's point---I think I am, judging by the other answers---but I think it might be a useful answer to other people who have a similar question and read this thread.
From Peter Norvig's site:
"Learn at least a half dozen programming languages. Include one language that supports class abstractions (like Java or C++), one that supports functional abstraction (like Lisp or ML), one that supports syntactic abstraction (like Lisp), one that supports declarative specifications (like Prolog or C++ templates), one that supports coroutines (like Icon or Scheme), and one that supports parallelism (like Sisal). "
http://norvig.com/21-days.html
I'm amazed that after 6 months and hundreds of votes, noone has mentioned SQL ...
In the types as theorems / advanced type systems: Coq ( I think Agda comes in this category too).
Coq is a proof assistant embedded into a functional programing language.
You can write mathematical proofs and Coq helps to build a solution.
You can write functions and prove properties about it.
It has dependent types, that alone blew my mind. A simple example:
concatenate: forall (A:Set)(n m:nat), (array A m)->(array A n)->(array A (n+m))
is the signature of a function that concatenates two arrays of size n and m of elements of A and returns an array of size (n+m). It won't compile if the function doesn't return that!
Is based on the calculus of inductive constructions, and it has a solid theory behind it.
I'm not smart enough to understand it all, but I think is worth taking a look, specially if you trend towards type theory.
EDIT: I need to mention: you write a function in Coq and then you can PROVE it is correct for any input, that is amazing!
One of the languages which i am interested for have a very different point of view (including a new vocabulary to define the language elements and a radical diff syntax) is J. Haskell would be the obvious choice for me, although it is a functional lang, cause its type system and other unique features open your mind and makes you rethink you previous knowledge in (functional) programming.
Just like fogus has suggested it to you in his list, I advise you too to look at the language OzML/Mozart
Many paradigms, mainly targetted at concurrency/multi agent programming.
Concerning concurrency, and distributed calculus, the equivalent of Lambda calculus (which is behind functionnal programming) is called the Pi Calculus.
I have only started begining to look at some implementation of the Pi calculus. But they already have enlarged my conceptions of computing.
Pict
Nomadic Pict
FunLoft. (this one is pretty recent, conceived at INRIA)
Dataflow programming, aka flow-based programming is a good step ahead on the road. Some buzzwords: paralell processing, rapid prototyping, visual programming (not as bad as sounds first).
Wikipedia's articles are good:
In computer science, flow-based
programming (FBP) is a programming
paradigm that defines applications as
networks of "black box" processes,
which exchange data across predefined
connections by message passing, where
the connections are specified
externally to the processes. These
black box processes can be reconnected
endlessly to form different
applications without having to be
changed internally. FBP is thus
naturally component-oriented.
http://en.wikipedia.org/wiki/Flow-based_programming
http://en.wikipedia.org/wiki/Dataflow_programming
http://en.wikipedia.org/wiki/Actor_model
Read JPM's book: http://jpaulmorrison.com/fbp/
(We've written a simple implementation in C++ for home automation purposes, and we're very happy with it. Documentation is under construction.)
You've learned a lot of languages. Now is the time to focus on one language, and master it.
perhaps you might want to try LabView for it's visual programming, although it's for engineering purposes.
nevertheless, you seem pretty interested in all that's out there, hence the suggestion
also, you could try the android appinventor for visually building stuff
Bruce A. Tate, taking a page from The Pragmatic Programmer wrote a book on exactly that:
Seven Languages in Seven Weeks: A Pragmatic Guide to Learning Programming Languages
In the book, he covers Clojure, Haskell, Io, Prolog, Scala, Erlang, and Ruby.
Mercury: http://www.mercury.csse.unimelb.edu.au/
It's a typed Prolog, with uniqueness types and modes (i.e. specifying that the predicate append(X,Y,Z) meaning X appended to Y is Z yields one Z given an X and Y, but can yield multiple X/Ys for a given Z). Also, no cut or other extra-logical predicates.
If you will, it's to Prolog as Haskell is to Lisp.
Programming does not cover the task of programmers.
New things are always interesting, but there are some very cool old stuff.
The first database system was dBaseIII for me, I was spending about a month to write small examples (dBase/FoxPro/Clipper is a table-based db with indexes). Then, at my first workplace, I met MUMPS, and I got headache. I was young and fresh-brained, but it took 2 weeks to understand the MUMPS database model. There was a moment, like in comics: after 2 weeks, a button has been switched on, and the bulb has just lighten up in my mind. MUMPS is natural, low level, and very-very fast. (It's an unbalanced, unformalized btree without types.) Today's trends shows the way back to it: NoSQL, key-value db, multidimensional db - so there are only some steps left, and we reach Mumps.
Here's a presentation about MUMPS's advantages: http://www.slideshare.net/george.james/mumps-the-internet-scale-database-presentation
A short doc on hierarchical db: http://www.cs.pitt.edu/~chang/156/14hier.html
An introduction to MUMPS globals (in MUMPS, local variables, short: locals are the memory variables, and the global variables, short: globals are the "db variables", setting a global variable goes to the disk immediatelly):
http://gradvs1.mgateway.com/download/extreme1.pdf (PDF)
Say you want to write a love poem...
Instead of using a hammer just because there's one already in your hand, learn the proper tools for the task: learn to speak French.
Once you've reached near-native speaking level, you're ready to start your poem.
While learning new languages on an academical level is an interesting hobby, IMHO you can't really learn to use one until you try to apply it to a real world problem. So, rather than looking for a new language to learn, I'd in your place first look for a new things to build, and only then I'd look for the right language to use for that one specific project. First pick the problem, then the tool, not the other way around..
For anyone who hasn't been around since the mid 80's, I'd suggest learning 8-bit BASIC. It's very low-level, very primitive and it's an interesting exercise to program around its holes.
On the same line, I'd pick an HP-41C series calculator (or emulator, although nothing beats real hardware). It's hard to wrap your brain around it, but well worth it. A TI-57 will do, but will be a completely different experience. If you manage to solve second degree equations on a TI-55, you'll be considered a master (it had no conditionals and no branches except a RST, that jumped the program back to step 0).
And last, I'd pick FORTH (it was mentioned before). It has a nice "build your language" Lisp-ish thing, but is much more bare metal. It will teach you why Rails is interesting and when DSLs make sense and you'll have a glipse on what your non-RPN calculator is thinking while you type.
PostScript. It is a rather interesting language as it's stack based, and it's quite practical once you want to put things on paper and you want either to get it done or troubleshoot why isn't it getting done.
Erlang. The intrinsic parallelism gives it a rather unusual feel and you can again learn useful things from that. I'm not so sure about practicality, but it can be useful for some fast prototyping tasks and highly redundant systems.
Try programming GPUs - either CUDA or OpenCL. It's just C/C++ extensions, but the mental model of the architecture is again completely different from the classic approach, and it definitely gets practical once you need to get some real number crunching done.
Erlang, Forth and some embedded work with assembly language. Really; buy an Arduino kit or something similar, and create a polyphonic beep in assembly. You'll really learn something.
There's also anic:
https://code.google.com/p/anic/
From its site:
Faster than C, Safer than Java, Simpler than *sh
anic is the reference implementation compiler for the experimental, high-performance, implicitly parallel, deadlock-free general-purpose dataflow programming language ANI.
It doesn't seem to be under active development anymore, but it seems to have some interesting concepts (and that, after all, is what you seem to be after).
While not meeting your requirement of "different" - I'd wager that Fantom is a language that a professional programmer should look at. By their own admission, the authors of fantom call it a boring language. It merely shores up the most common use cases of Java and C#, with some borrowed closure syntax from ruby and similar newer languages.
And yet it manages to have its own bootstrapped compiler, provide a platform that has a drop in install with no external dependencies, gets packages right - and works on Java, C# and now the Web (via js).
It may not widen your horizons in terms of new ways of programming, but it will certainly show you better ways of programming.
One thing that I see missing from the other answers: languages based on term-rewriting.
You could take a look at Pure - http://code.google.com/p/pure-lang/ .
Mathematica is also rewriting based, although it's not so easy to figure out what's going on, as it's rather closed.
APL, Forth and Assembly.
Have some fun. Pick up a Lego Mindstorm robot kit and CMU's RobotC and write some robotics code. Things happen when you write code that has to "get dirty" and interact with the real world that you cannot possibly learn in any other way. Yes, same language, but a very different perspective.
I see a lot of the word 'expressiveness' when people want to stress one language is better than the other. But I don't see exactly what they mean by it.
Is it the verboseness/succinctness? I mean, if one language can write down something shorter than the other, does that mean expressiveness? Please refer to my other question - Article about code density as a measure of programming language power
Is it the power of the language? Paul Graham says that one language is more powerful than the other language in a sense that one language can do that the other language can't do (for example, LISP can do something with macro that the other language can't do).
Is it just something that makes life easier? Regular expression can be one of the examples.
Is it a different way of solving the same problem: something like SQL to solve the search problem?
What do you think about the expressiveness of a programming language? Can you show the expressiveness using some code?
What's the relationship with the expressiveness and DSL? Do people come up with DSL to get the expressiveness?
Personally, I feel that the "expressiveness" of a language really comes down to how clearly the language constructs can "express" the developer's intentions.
For example, I feel that C# (especially LINQ via C# 3+) is becoming much more expressive. This LINQ statement is a great example:
var results = collection.Where(item => item > 5);
Without knowing the details of the language or the implementation being used, the developer intent is (in my opinion) very clear in the above statement.
I do not think that the verboseness of the language is equal to its expressiveness, however, there is some correlation in place. If a language requires a lot of code in order to express an abstraction, it is less expressive. These are two related, but different, concepts.
The same is true with power - although here a language's features (ie: power) must be complete enough to express the abstraction clearly. Without this, expressiveness will suffer. That being said, a language can be very "powerful" in terms of features, but not necessarily be expressive, if the feature set is difficult to understand.
"Expressiveness" means the ability to say only what you want done:
bad_event = events.find(&:bad)
rather than how you want it done:
i = 0
bad_event = nil
while i < events.size && bad_event.nil?
event = events[i]
if event.bad?
bad_event = event
end
i += 1
end
Among the things that contribute to expressiveness are:
A lack of required syntactic sugar
First-class functions
Garbage collection
Either dynamic typing or type inference
The language core not being slavishly minimalistic
Good functionality in the standard library
To some degree, the expressiveness of any language can be increased by shoving as much "how to do it" off into subroutines/objects as possible so that most of the remaining code is "what to do." The amount of "how to do it" code needed in the most abstract code is one measure of a language's expressiveness: The more the code looks like pseudocode, the more expressive it is of the programmer's intent.
One can also think about the "meta-expressiveness" of a language: How expressive is the language at constructing Domain Specific Languages?
I like Matthias Felleisen's notion of expressive power, which is comparative:
Language A is strictly more expressive than language B if both of the following are true:
Any program written in language B can be rewritten in language A while keeping the essential structure of the program intact.
Some programs written in language A have to be violently restructured in order to be written in language B.
Usually we want to make these comparisons by looking at some kind of "essential core" of a language—for example, maybe we want to consider a dialect of C with only while and not also for and do...while. Or maybe we want to consider a dialect of Perl with only a prefix if form and no unless form. But sometimes these superficial syntactic distinctions are exactly what we mean by "expressive power"; to some programmers it's important to say
die ("found no solutions") unless length(solutions) > 0;
instead of
if (length(solutions) == 0) { die("found no solutions"); }
So you have to establish whether you're asking about expressive power of surface syntax or deeper structure.
The other thing I like about Felleisen's idea is that it admits of the notion of two languages which are definitely different, but neither is more expressive than the other.
You can read a more detailed exposition in the first two pages of his paper On the Expressive Power of Programming Languages. After that comes a lot of pointy-headed theory :-)
If you want an answer that's somewhat theoretical but more rigorous than most, you might want to look around for Matthias Felleisen's On the Expressive Power of Programming Languages. I'm pretty sure a bit of looking around the net will turn up at least a few copies.
If you want a more practical answer of what most people actually mean when they say it, that's, frankly, rather different. At least in my experience, an "expressive" language usually means: "I like the language, but can't cite much (if any) objective support for doing so." Conversely, things like "less expressive", or "not expressive" generally mean: "I don't like the language (or like it less), but can't cite much (if any) objective support for doing so."
"Not expressive" is often similar to a politician accusing another of being "fascist" -- clearly pejorative, but without any meaningful definition of what's supposedly wrong.
One of the big problems stems from a fundamental difference of opinion. There are at least two fundamentally different general ideas that people seem to have about expressiveness:
the ability to express a wide variety of ideas.
the ability to express some specific ideas clearly (and often succinctly).
To consider some extreme examples, assembly language would qualify as highly expressive by the first criteria--you can do essentially anything in assembly language that you can in a higher level language, and you can do some things in assembly language that you can't in essentially any higher level language.
Obviously, assembly language doesn't look nearly so good by the second measure--it typically requires quite a large amount of fairly opaque code to accomplish much. This measure would tend to favor a language like Haskell or APL, to give only a couple of examples.
These two notions of what "expressive" means are frequently close to diametrically opposed. The first tends to favor the "lowest" level languages, while the second tends to favor the "highest" level. At least from what I've seen, most people really start from the language they like, and their definition of "expressive" is basically whatever balance of the two criteria is required for their preferred language to be the "best".
For me, it is the ability of the language to clearly express my logic and ideas through code, in a way that somebody else reading the code can easily figure out what I was thinking when I wrote it.
Wikipedia has a bit about the concept. I myself take it to mean that a language can accomplish more with less (the so called "informal usage" in the Wikipedia article).
I consider JavaScript expressive (though this could be because Douglas Crockford drilled that idea into my noggin) because it can do so much with just a few keywords. For instance, the function keyword is a function, as well as a method, a class, and a lambda.
Some code illustration (leaving out some details for brevity) in JavaScript. It's an event class I wrote:
SJJS.util.Event = (function() {
var _listeners = [];
var _listenerReturns = [];
return {
addDomListener: function(element, eventName, listener) {
},
trigger: function(element, eventName) {
},
removeListener: function(eventlistener) {
}
}
})();
With just function, var, and some curly braces and parentheses I made a static class with methods and private variables.
Generally speaking, with a programming language which is turing complete you can do anything that another turing complete language can do. That being said, some can do it a lot better than other.
I take expressiveness to mean how much you can say easily, and how well / clearly it can be said. The ability to be terse is part of that ( a very powerful and terse language is one like J ). Generally I find that being concise is a good marker of being expressive. If the language can express a complex operation in a simple manner, it's going in the proper direction.
As to the power, expressiveness isn't all the power of a language. While it may be part of it, speed, security, stability, all of those things factor in as well.
example: summation of a list in Common lisp using the loop operator is concise and expressive
(loop for x in list sum x)
Precision, concision, and readability are the primary components in expressiveness.
I always took it to be roughly equivalent to how high-level a language is. If you wanted to try to quantify expressiveness, the units would be something like "machine code instructions per language statement"
A more expressive language might be very good at doing rather a lot of work without writing a lot of code. However, it would probably be more domain-specific and a wee bit slower for some tasks than a less expressive one.
From Wikipedia: In computer science, the expressive power (also called expressiveness or expressivity) of a language is the breadth of ideas that can be represented and communicated in that language. The more expressive a language is, the greater the variety and quantity of ideas it can be used to represent.
So, I agree. "How easy, comprehensive and composable the language for you to express your intents.": I believe, this is the measure of expressiveness.
QUESTION: Is it the verboseness/succinctness? I mean, if one language can write down something shorter than the other, does that mean expressiveness?
No. For example, is Brainfuck language expressive? I don't think so. Look at an Hello World example in Brainfuck:
++++++++++[>+++++++>++++++++++>+++>+<<<<-]>++.>+.+++++++..+++.>++.<<+++++++++++++++.>.+++.------.--------.>+.>.
Or: hq9plus language. Hello World code:
H
QUESTION: Is it the power of the language? Paul Graham says that one language is more powerful than the other language in a sense that one language can do that the other language can't do (for example, LISP can do something with macro that the other language can't do).
I disagree with Paul. As you see in the above examples, hq9plus language is doing Hello World with one letter: H. Whereas, most of the other languages will do it with much more letters. But, you can create composable and easy to read code with other languages. If hq9plus is doing Hello World with H, does it mean that is it powerful? I believe no.
QUESTION: Is it just something that makes life easier? Regular expression can be one of the examples.
Regexes are great, but sometimes they lose their expressive power. Sometimes, it depends on the programmer.
QUESTION: Is it a different way of solving the same problem: something like SQL to solve the search problem?
Half Yes. SQL is a declarative and very expressive language. Because the underlying engines and technologies can advance and change without you to change your SQL queries. This makes it very expressive. There are many queries have been working for decades and the underlying technology of databases change. But, your queries don't need to. I think this is due to the power of its expressiveness.
I also believe that functional languages are very expressive. Because, you only describe your intent, not your hows, and the underlying technologies can always change and optimize, but, it won't hurt your expressive code.
Example:
// top 10 products with rating higher than 5
return products
.sort(p => p.rating)
.filter(p => p.rating > 5)
.map(p => p.title)
.take(10)
Above program is expressive, it conveys your intents. And, most probably it won't change when the underlying mechanisms change.
Take for example LINQ. It allows you to use functional programming.
Functional programming emphasizes the application of functions, in contrast to the imperative programming style, which emphasizes changes in state.
LINQ allows your to express what you want done instead of how to do it. This is a clear example of expressiveness.