Related
I have just inherited a project that uses SecureSWF. I am trying to utilize RobotLegs (which uses SwiftSuspenders for reflection to implement dependency injection) and have just discovered that SecureSWF breaks the build. Has anyone had a similar problem? Is there a workaround? Is it possible to obscure a SWF that's built with RobotLegs at all?
It's straightforward, actually. You need NAMES for reflection. And they are the primary target for ANY kind of obfuscation and mangling. Since we absolutely can not abuse the verify mechanism in flash player VM (which is damn good), we have no way in getting around it.
I'm using secureSWF too, and I have a mechanism of sewing skins and controllers together with descrybeType() and a hell of a lot of checking of types and members. I exclude my sensitive to obfuscation classes from the protection workflow. They are of no use to a hacker anyway.
I heard from someone that DSL is really powerful in some specific fields. So i want to find out if i can put it into my skill sets.
The first problem came out is What is DSL exactly? After doing some search, it seems Groovy supports DSL very well. Then i go and read Groovy's documents and try it out by myself.
And i got the impression that DSL is just some kind of configuration files consisting of texts, XMLs and you use some tools like Groovy to parse it, it magically become some methods or functions you can invoke. What happened?
I read something, but cannot get it straight. Any Help?
Did you read this? Martin Fowler is an authority on the subject and a great writer. I doubt that anyone will improve on the first paragraph. If you still don't get it, give it some time and re-read the article a few times.
I'd recommend looking into JetBrain's MPS
A book might be overwhelming, but there's a relatively new one available.
And i got the impression that DSL is just some kind of configuration
files consisting of texts, XMLs and you use some tools like Groovy to
parse it, it magically become some methods or functions you can
invoke. What happened?
I don't think your impression is entirely accurate. I'd forget about Groovy and parsing and all the implementation details for now. Focus on the problem that DSL is trying to solve.
A DSL designer tries to come up with a pseudo programming language that an expert, who is unfamilar with programming languages like Groovy or Java or C#, would recognize as a simple language describing they way they solve problems.
The DSL uses terms and concepts familiar to any one knowledgable about that domain.
The DSL shields users from the underlying implementation details so they can focus on how to attack their problems.
A DSL is written for the convenience of business users, not developers.
Keep that in mind and the rest is implementation. Eye on the prize....
A domain specific language (DSL) is a programing language that is not fully featured. The point is that programing in a DSL can be easier than programing in a general purpose language, and be less prone to bugs. The "domain" in "domain specific language" refers to the specific purpose the language will be used for.
For example, the language that a calculator uses with just + - * / and numbers could be called a domain specific language. It has the advantage over a regular programing language in that programs will never segfault, crash, loop forever, etc. Other examples of domains might be web development -- for example, Ur/Web is a DSL for building web applications. SQL is a database domain specific language. etc.
I don't know much about Groovy, but it seems that there are particular tools for using it to create DSLs. Fundamentally, to create a DSL you need to specify a syntax, along with some sort of semantics. How exactly Groovy does this I do not know.
DSL is a language dedicated to a specific domain. For instance, the well-known CSS is a Domain Specific Language serving the look and formatting of a document.
By using Groovy you might create your own DSL focusing on any selected domain - e.g. accounting, telecommunications, banking etc. This means, that the language will use the common terminology of this area meeting the needs of this domain. This language will be easily understood by people of this domain that are not necessarily technical (e.g. accountants). In some times, it focuses on being used by non-programmers. Especially Groovy is a dynamic language with which you can enable end-users to add code scripts dynamically similarly to what Excel does with VB, through configuration files.
You should delve into Martin Fowler's publications if you are interested in this subject, anyway.
I would like to know which approach is faster, using the pure PHP in the HTML files or using a template engines like Smarty,Twig, ...
What I would particularly like to know is next: which is parsed faster, is the Smarty cache for example faster than using pure PHP?
Which of the template engines is the fastest? I'm about to rewrite simple application where speed is on the first place.
"Depends" is the answer to all your questions.
What is "faster"? Execution time? Development time? Maintenance? Memory overhead? A mixture of them? A template engine is usually trading in some performance (speed, memory) for better development and maintenance.
If you are talking about purely dynamic templating (meaning: template evaluated on every request) PHP will outrun any template engine. This is a nobrainer, really. If you're taking caching into account, a template engine like Smarty may help. Caching is nothing you couldn't implement yourself in plain PHP, though. With Smarty it's just been done for you (and on a far more sophisticated level than you possibly would).
If you are using a framework, say Symfony, it might be wise to use Twig, as Twig and Symfony are tightly integrated. Sure you can use Smarty or plain PHP. The question here is: is it practicable?
Caching makes sense when building sites from datasources like a database or remote APIs. What you are really saving (in a sense of reducing) here are database calls, intensive calculations, etc. Check if you have any time-intensive functions running to build your site. If so, use caching (if you can).
Knowing development/maintenance/convenience/performance trade-offs, I would (always) recommend using a template engine. Being a Smarty developer, I'll, of course, suggest using Smarty. That is unless you're using Symfony, then you might be better of with Twig. Or some other framework featuring some other template engine.
Please ignore posts like Smarty vs. Twig, as they only compare a very limited view of the engines. Don't trust benchmarks you haven't faked yourself™.
In general, though, Smarty 3.1 is a bit faster than Twig. Twig is doing a lot of stuff at runtime (being the time when a template is executed) that Smarty does on compile time (being the time when a template is prepared for execution). Twig is not really pissing away speed here. Twig needs to do certain stuff at runtime by design. They traded a bit of performance for a bit of "convenience" (Accessing arrays and objects with the same notation, for example).
Let's tear the tropes related to this subject apart:
1. Keep logic out of the presentation - Do not put 'code' into your HTML
Anyone who says this and then tells you to go with templating is contradictory:
PHP is an interpreted language - it becomes C code on execution.
The templating 'syntax' is interpreted into PHP
They must stop lying to themselves. Their 'templating syntax' is a programming language built on top of another, which in turn is built on top yet another language - That's inefficient, redundant, and weird.
Furthermore, I fail to see how the very existence of the variables every templating engine that ever was depends on aren't considered logic - Their existence, content and implementation depend on a logical backend.
And what of those templating systems with if/else statements and for loops? That's the very essence of logic - The very concepts which most programming languages utilize. They require variable data which can only be generated or exist through some form of computation.
You cannot serve dynamic content without mixing presentation with logic. It's impossible.
2.1 It's safer...
So, you don't trust your HTML guy?
Case: You think your HTML/CSS guy is stupid and will accidentally print the database password
If that's so, I've got news for you - Your environment is already not safe if sensitive data can be accessed/modified from anywhere within the program.
Case: You think your HTML guy will print random server constants - it's dangerous to allow him, as an individual, to work with server logic
I see - He's either stupid, or hates his job and wants to be fired and therefore will do something dumb like printing session variables. Fine, but to that I'll say...
...Why the heck is this stuff not peer reviewed? Even if he had no access to direct server logic but rather a fancy templating system, he could still equally spread his his stupidity/hatred merely because he has final say on output. Or, he could even be in cahoots with another programmer (If any) and still access server constants and co.
-
2.2.1 Good templating engines automatically sanitize output, or allow the templating-guy to do it himself - he knows better when data should be sanitized
You dummy.
You don't know when output should be sanitized? You couldn't do that yourself..?
Even so, maybe you're just the code monkey and the HTML guy is a web-security HTML-injection specialist, and he should be the one sanitizing output. In that case, giving him access to PHP also allows him to use the likes of htmlspecialchars() rather than whatever the template gives him to do the same thing.
Regarding automatic escaping, provided you're safely passing along content, you can implement such a simple feature within the code you're doing so.
--
2.2 ...and I can control what data is being worked with
Think about classes, functions, etc - You throw data in, they work with it, then you get a result. Typically they do not deal with outside data unless it is handed to them (Doing otherwise is unclear, dangerous and bad practice - Some constants aside). Through these same methods, you can pass on precisely what you need to your output in an efficient, clear and unlimited manor.
--
All that said, it seems like the reason you think your templating engine is any safer than plain code is because you're lacking in several areas of general safety:
You (Or whoever) do not peer review content - You allow individuals to output content.
You are not implementing proper or safe programming practices, and seem to not realize that you can control what's passed along from point A to B.
3. PHP syntax is too hard/difficult to teach the style people
The truth is it's no more complicated than the psuedo-syntax created by template systems such as Smarty, so if this is an issue than dynamic content isn't for you.
The following is in PHP 'short syntax' - Is it too difficult?
<div class='username'><?= $username ?></div>
4. It's too much work to develop my own solution
Though I'd argue it's not, you're free to choose whatever you wish! Choose whatever fits your needs best. They're usually free, not difficult to integrate, and come with loads of features out of the box.
I'm under the impression that most people opt for templating simply because it looks 'neater' within the file - They love thinking that the TPL file is some special thing they created, they like the way the syntax looks; As if by some magic, the variable is 'called' by the little # or # symbol and hops from your logic into the output.
It seems like a trick - The beautiful enchantress (AKA The templating engine) draws you in with her beauty. Though she's appealing to the eye, she's really a blood sucking demon and extracts your soul (Server resources) in exchange for eye candy nobody else sees (Your users would much rather have a faster website and more features funded by the $$$ you're saving on power/server renting)
<title>{{#title}}</title>
Vs
<title><?= $title ?></title>
I will admit, there's only one case I can think of in which templates have any ground over PHP - Portability to other applications. appartisan's answer addresses that. Even so, it's not hard to replace <?= $var ?> with {{#var}} - That's a job for a templating-esque system.
Simply and purely opinion, I think the only advantage is portability. You can re-use templates or views from a template engine into other backend application. Say you're moving your application from PHP to Java, you don't need to refactor the templates.
Otherwise, you're adding complexity, adding other layer of execution ( more time ), more requirements to maintain the application ( you need people that knows that template engine ), and so on. PHP itself it's the best and more featured template engine you're going to get, probably the fastest, and you can do caching also, with the advantage of controlling cache from the backend application, and not from the view.
I will take up this again as things have changed significantly and there are some pieces of evidence missing from the earlier answer.
Without getting deep into why frameworks use template engines over PHP which most do. For some reason there is a constant effort to "fix" PHP with another abstraction layer. Always with claims of simplicity without loss of versatility or performance.
Regardless, the use of PHP is still the fastest and most versatile way of templating. PHP in it's earliest incarnations looked much like a templating language. But let's take a look at the advancements in PHP and place them side by side with the after layers.
Twig and some others claim caching something which was always an addon in earlier versions of PHP. Caching is now a default part of PHP5.5+ (Opcache) and so using PHP as a template language will give more performance enhancements.
Twig and others claim simple syntax for designers. In comparing the syntax of a template engine you'll see that the logic is similar with the only benefit of using a template system like Twig being another layer of security separation between the designer and the underlying system code.
Two very popular CMS Wordpress and Drupal used PHP as their template engines. So the old argument of using a template engine to secure and simplify the use of PHP while designing a website is not really valid in today's web. While Drupal 8 is moving on to Twig it mostly because twig is part of Symfony Framework ( returning to why do frameworks use template engines). Wordpress on the other hand is still using PHP. As Wordpress is growing by leaps and bounds with web designers using PHP to help this happen. Drupals Community has also been split in part by decisions to use Twig and Symfony.
So it would seem that using PHP is the better choice in terms of performance but also the preference for themers and designers going forward. At least all evidence leads to this conclusion.
That being said here's my baseless opinion. I think that using anything other than PHP as template engine in today's web covers some inherent weaknesses in the underlying framework or web application architecture. That weakness being its complexities and complications that cannot be explained easily at the designer or themer level.
If you are writing a lightweight application that has to be small. Keep it small and performing optimally by using PHP and leave the other engines to "enterprise" level groups and projects
I have a problem with the argument that logic and data display must be separared as much as possible. I found that data validation and display actually requires a lot of logic on forms. Information about data type, number range, relation between different data requires a lot of code. The real question is should we use a template language on the server side or Javascript on the client side. By using Ajax and client side code for data display and validation, I end up having very little template code. The biggest problem with template engines is the intoduction of new code rules and syntax. I see the future with PHP, Jquery and Ajax and template engines loosing its appeal.
I have encountered this topic lately and couldn't understand why they are needed.
Can you explain why I should use them in my projects and how they can ease my life.
Examples will be great, and where from I can learn this topic little more.
At least you have framed the question from the correct perspective =)
The usual reasons for using a code generator are given as productivity and consistency because they assume that the solution to a consistent and repetitive problem is to throw more code at it. I would argue that any time you are considering code generation, look at why you are generating code and see if you can solve the problem through other means.
A classic example of this is data access; you could generate 250 classes ( 1 for each table in the schema ) effectively creating a table gateway solution, or you could build something more like a domain model and use NHibernate / ActiveRecord / LightSpeed / [pick your orm] to map a rich domain model onto the database.
While both the hand rolled solution and ORM are effectively code generators, the primary difference is when the code is generated. With the ORM it is an implicit step that happens at run-time and therefore is one-way by it's nature. The hand rolled solution requires and explicit step to generate the code during development and the likelihood that the generated classes will need customising at some point therefore creating problems when you re-generate the code. The explicit step that must happen during development introduces friction into the development process and often leads to code that violates DRY ( although some argue that generated code can never violate DRY ).
Another reason for touting code generation comes from the MDA / MDE world ( Model Driven Architecture / Engineering ). I don't put much stock in this but rather than providing a number of poorly expressed arguments, I'm simply going to co-opt someone elses - http://www.infoq.com/articles/8-reasons-why-MDE-fails.
IMHO code generation is the only solution in an exceedingly narrow set of problems and whenever you are considering it, you should probably take a second look at the real problem you are trying to solve and see if there is a better solution.
One type of code generation that really does enhance productivity is "micro code-generation" where the use of macros and templates allow a developer to generate new code directly in the IDE and tab / type their way through placeholders (eg namespace / classname etc). This sort of code generation is a feature of resharper and I use it heavily every day. The reason that micro-generation benefits where most large scale code generation fails is that the generated code is not tied back to any other resource that must be kept in sync and therefore once the code is generated, it is just like all the other code in the solution.
#John
Moving the creation of "basic classes" from the IDE into xml / dsl is often seen when doing big bang development - a classic example would be developers try to reverse engineer the database into a domain model. Unless the code generator is very well written it simply introduces an additional burden on the developer in that every time they need to update the domain model, they either have to context-switch and update the xml / dsl or they have to extend the domain model and then port those changes back to the xml / dsl ( effectively doing the work twice).
There are some code generators that work very well in this space ( the LightSpeed designer is the only one I can think of atm ) by acting as the engine for a design surface but often
these code generators generate terrible code that cannot be maintained (eg winforms / webforms design surfaces, EF1 design surface) and therefore rapidly undo any productivity benefits gained from using the code generator in the first place.
Well, it's either:
you write 250 classes, all pretty much the same, but slightly different, e.g. to do data access; takes you a week, and it's boring and error-prone and annoying
OR:
you invest 30 minutes into generating a code template, and let a generation engine handle the grunt work in another 30 minutes
So a code generator gives you:
speed
reproducability
a lot less errors
a lot more free time! :-)
Excellent examples:
Linq-to-SQL T4 templates by Damien Guard to generate one separate file per class in your database model, using the best kept Visual Studio 2008 secret - T4 templates
PLINQO - same thing, but for Codesmith's generator
and countless more.....
Anytime you need to produce large amounts of repetetive boilerplate code, the code generator is the guy for the job. Last time I used a code generator was when creating a custom Data Access Layer for a project, where the skeleton for various CRUD actions was created based on an object model. Instead of hand-coding all those classes, I put together a template-driven code generator (using StringTemplate) to make it for me. The advandages of this procedure was:
It was faster (there was a large amount of code to generate)
I could regenerate the code in a whim in case I detected a bug (code can sometimes have bugs in the early versions)
Less error prone; when we had an error in the generated code it was everywhere which means that it was more likely to be found (and, as noted in the previous point, it was easy to fix it and regenerate the code).
Using GUI builders, that will generate code for you is a common practice. Thanks to this you don't need to manually create all widgets. You just drag&drop them and the use generated code. For simple widgets this really saves time (I have used this a lot for wxWidgets).
Really, when you are using almost any programming language, you are using a "code generator" (except for assembly or machine code.) I often write little 200-line scripts that crank out a few thousand lines of C. There is also software you can get which helps generate certain types of code (yacc and lex, for example, are used to generate parsers to create programming languages.)
The key here is to think of your code generator's input as the actual source code, and think of the stuff it spits out as just part of the build process. In which case, you are writing in a higher-level language with fewer actual lines of code to deal with.
For example, here is a very long and tedious file I (didn't) write as part of my work modifying the Quake2-based game engine CRX. It takes the integer values of all #defined constants from two of the headers, and makes them into "cvars" (variables for the in-game console.)
http://meliaserlow.dyndns.tv:8000/alienarena/lua_source/game/cvar_constants.c
Here is the short Bash script which generated that code at compile-time:
http://meliaserlow.dyndns.tv:8000/alienarena/lua_source/autogen/constant_cvars.sh
Now, which would you rather maintain? They are both equivalent in terms of what they describe, but one is vastly longer and more annoying to deal with.
The canonical example of this is data access, but I have another example. I've worked on a messaging system that communicates over serial port, sockets, etc., and I found I kept having to write classes like this over and over again:
public class FooMessage
{
public FooMessage()
{
}
public FooMessage(int bar, string baz, DateTime blah)
{
this.Bar = bar;
this.Baz = baz;
this.Blah = blah;
}
public void Read(BinaryReader reader)
{
this.Bar = reader.ReadInt32();
this.Baz = Encoding.ASCII.GetString(reader.ReadBytes(30));
this.Blah = new DateTime(reader.ReadInt16(), reader.ReadByte(),
reader.ReadByte());
}
public void Write(BinaryWriter writer)
{
writer.Write(this.Bar);
writer.Write(Encoding.ASCII.GetBytes(
this.Baz.PadRight(30).Substring(0, 30)));
writer.Write((Int16)this.Blah.Year);
writer.Write((byte)this.Blah.Month);
writer.Write((byte)this.Blah.Day);
}
public int Bar { get; set; }
public string Baz { get; set; }
public DateTime Blah { get; set; }
}
Try to imagine, if you will, writing this code for no fewer than 300 different types of messages. The same boring, tedious, error-prone code being written, over and over again. I managed to write about 3 of these before I decided it would be easier for me to just write a code generator, so I did.
I won't post the code-gen code, it's a lot of arcane CodeDom stuff, but the bottom line is that I was able to compact the entire system down to a single XML file:
<Messages>
<Message ID="12345" Name="Foo">
<ByteField Name="Bar"/>
<TextField Name="Baz" Length="30"/>
<DateTimeField Name="Blah" Precision="Day"/>
</Message>
(More messages)
</Messages>
How much easier is this? (Rhetorical question.) I could finally breathe. I even added some bells and whistles so it was able to generate a "proxy", and I could write code like this:
var p = new MyMessagingProtocol(...);
SetFooResult result = p.SetFoo(3, "Hello", DateTime.Today);
In the end I'd say this saved me writing a good 7500 lines of code and turned a 3-week task into a 3-day task (well, plus the couple of days required to write the code-gen).
Conclusion: Code generation is only appropriate for a relatively small number of problems, but when you're able to use one, it will save your sanity.
A code generator is useful if:
The cost of writing and maintaining the code generator is less than the cost of writing and maintaining the repetition that it is replacing.
The consistency gained by using a code generator will reduce errors to a degree that makes it worthwhile.
The extra problem of debugging generated code will not make debugging inefficient enough to outweigh the benefits from 1 and 2.
For domain-driven or multi-tier apps, code generation is a great way to create the initial model or data access layer. It can churn out the 250 entity classes in 30 seconds ( or in my case 750 classes in 5 minutes). This then leaves the programmer to focus on enhancing the model with relationships, business rules or deriving views within MVC.
The key thing here is when I say initial model. If you are relying on the code generation to maintain the code, then the real work is being done in the templates. (As stated by Max E.) And beware of that because there is risk and complexity in maintaining template-based code.
If you just want the data layer to be "automagically created" so you can "make the GUI work in 2 days", then I'd suggest going with a product/toolset which is geared towards the data-driven or two-tier application scenario.
Finally, keep in mind "garbage in=garbage out". If your entire data layer is homogeneous and does not abstract from the database, please please ask yourself why you are bothering to have a data layer at all. (Unless you need to look productive :) )
How 'bout an example of a good use of a code generator?
This uses t4 templates (a code generator built in to visual studio) to generate compressed css from .less files:
http://haacked.com/archive/2009/12/02/t4-template-for-less-css.aspx
Basically, it lets you define variables, real inheritance, and even behavior in your style sheets, and then create normal css from that at compile time.
Everyone talks here about simple code generation, but what about model-driven code generation (like MDSD or DSM)? This helps you move beyond the simple ORM/member accessors/boilerplate generators and into code generation of higher-level concepts for your problem domain.
It's not productive for one-off projects, but even for these, model-driven development introduces additional discipline, better understanding of employed solutions and usually a better evolution path.
Like 3GLs and OOP provided an increase in abstraction by generating large quantities of assembly code based on a higher level specification, model-driven development allows us to again increase the abstraction level, with yet another gain in productivity.
MetaEdit+ from MetaCase (mature) and ABSE from Isomeris (my project, in alpha, info at http://www.abse.info) are two technologies on the forefront of model-driven code generation.
What is needed really is a change in mindset (like OOP required in the 90's)...
I'm actually adding the finishing touches to a code generator I'm using for a project I've been hired on. We have a huge XML files of definitions and in a days worth of work I was able to generate over 500 C# classes. If I want to add functionality to all the classes, say I want to add an attribute to all the properties. I just add it to my code-gen, hit go, and bam! I'm done.
It's really nice, really.
There are many uses for code generation.
Writing code in a familiar language and generating code for a different target language.
GWT - Java -> Javascript
MonoTouch - C# -> Objective-C
Writing code at a higher level of abstraction.
Compilers
Domain Specific Languages
Automating repetitive tasks.
Data Access Layers
Initial Data Models
Ignoring all preconceived notions of code-generation, it is basically translating one representation (usually higher level) to another (usually lower level). Keeping that definition in mind, it is a very powerful tool to have.
The current state of programming languages has by no means reached its full potential and it never will. We will always be abstracting to get to a higher level than where we stand today. Code generation is what gets us there. We can either depend on the language creators to create that abstraction for us, or do it ourselves. Languages today are sophisticated enough to allow anybody to do it easily.
If with code generator you also intend snippets, try the difference between typing ctor + TAB and writing the constructor each time in your classes. Or check how much time you earn using the snippet to create a switch statement related to an enum with many values.
If you're paid by LOC and work for people who don't understand what code generation is, it makes a lot of sense. This is not a joke, by the way - I have worked with more than one programmer who employs this technique for exactly this purpose. Nobody gets paid by LOC formally any more (that I know of, anyway), but programmers are generally expected to be productive, and churning out large volumes of code can make someone look productive.
As an only slightly tangential point, I think this also explains the tendency of some coders to break a single logical unit of code into as many different classes as possible (ever inherit a project with LastName, FirstName and MiddleInitial classes?).
Here's some heresy:
If a task is so stupid that it can be automated at program writing time (i.e. source code can be generated by a script from, let's say XML) then the same can also be done at run-time (i.e. some representation of that XML can be interpreted at run-time) or using some meta-programming. So in essence, the programmer was lazy, did not attempt to solve the real problem but took the easy way out and wrote a code generator. In Java / C#, look at reflection, and in C++ look at templates
What's the definition of a Shim?
Simple Explanation via Cartoon
Summary
A shim is some code that takes care of what's asked (by 'interception'), without anyone being any wiser about it.
Example of a Shim
An example of a shim would be rbenv (a ruby tool). Calls to ruby commands are "shimmed". i.e. when you run bundle install, rbenv intercepts that message, and reroutes it according to the specific version of Ruby you are running. If that doesn't make sense try this example, or just think of the fairy god mother intercepting messages and delivering apposite outcomes.
That's it!
Important Clarifications on this example
Note: Like most analogies, this is not perfect: usually Ralph will get EXACTLY what he asked for - but the mechanics of HOW it was obtained is something Ralph doesn't care about. If Ralph asks for dog food, a good shim will deliver dog food.
I wanted to avoid semantic arguments, and complexity e.g. adapter gang of four design patterns, facade, proxy patterns - not that great when you're trying to explain a concept. Introducing code? Pedagogically risky. Wikipedia-like explanation? Boooring, too complex, and time consuming: so I had to deliberately simplify to a cartoon, so you can easily understand in a "fun" way, in 30 seconds, is memorable so you can move on. This approach is not for everyone: if you want a precise definition consider the Wikipedia entry on shims.
The term "shim" as defined in Wikipedia would technically be classified, based on its definition, as a "Structural" design pattern. The many types of “Structural” design patterns are quite clearly described in the (some would say defacto) object oriented software design patterns reference "Design Patterns, Elements of Reusable Object-Oriented Software" better known as the "Gang of Four".
The "Gang of Four" text outlines at least 3 well established patterns known as, "Proxy", "Adapter" and "Facade" which all provide “shim” type functionality. In most fields it’s often times the use and or miss use of different acronyms for the same root concept that causes people confusion. Using the word “shim” to describe the more specific “Structural” design patterns "Proxy", "Adapter" and "Facade" certainly is a clear example of this type of situation. A "shim" is simply a more general term for the more specific types of "Structural" patterns "Proxy", "Adapter", "Facade" and possibly others.
According to Microsoft's article "Demystifying Shims":
It’s a metaphor based on the English language word shim, which is an
engineering term used to describe a piece of wood or metal that is
inserted between two objects to make them fit together better. In
computer programming, a shim is a small library which transparently
intercepts an API, changes the parameters passed, handles the
operation itself, or redirects the operation elsewhere. Shims can also
be used for running programs on different software platforms than they
were developed for.
So a shim is a generic term for any library of code that acts as a middleman and partially or completely changes the behavior or operation of a program. Like a true middleman, it can affect the data passed to that program, or affect the data returned from that program.
The Windows API is an example:
The application is generally unaware that the request is going to a
shim DLL instead of to Windows itself, and Windows is unaware that the
request is coming from a source other than the application (because
the shim DLL is just another DLL inside the application’s process).
So the two programs that make the "bread" of the "shim sandwich" should not be able to differentiate between talking to their counterpart program and talking to the shim.
What are some pros and cons of using shims?
Again, from the article:
You can fix applications without access to the source code, or without
changing them at all. You incur a minimal amount of additional
management overhead... and you can fix a
reasonable number of applications this way. The downside is support as
most vendors don’t support shimmed applications. You can’t fix every
application using shims. Most people typically consider shims for
applications where the vendor is out of business, the software isn’t
strategic enough to necessitate support, or they just want to buy some
time.
As for origins of the word, quoth Apple's Dictionary widget
noun
a washer or thin strip of material used to align parts,
make them fit, or reduce wear.
verb ( shimmed, shimming) [ trans. ]
wedge (something) or fill up (a space) with a shim.
ORIGIN early 18th cent.: of unknown origin
This seems to fit quite well with how web designers use the term.
Shims are used in .net 4.5 Microsoft Fakes framework to isolate your application from other assemblies for unit testing. Shims divert calls to specific methods to code that you write as part of your test
As we could see in many responses here, a shim is a sort of adapter that provides functionality at API level which was not necessarily part of that API. This thread has a lot of good and complete responses, so I'm not expanding the definition further.
However, I think I can add a good example, which is the Javascript ES5 Shim (https://github.com/es-shims/es5-shim):
Javascript has evolved a lot during the last few years, and among many other changes to the language specification, a lot of new methods have been added to its core objects.
For example, in the ES2015 specification (aka ES5), the method find has been added to the Array prototype. So let's say you are running your code using a JavasScript engine prior to this specification (ex: Node 0.12) which doesn't offer that method yet. By loading the ES5 shim, these new methods will be added to the Array prototype, allowing you to make use of them even if you are not running on a newer JavaScript specification.
You might ask: why would someone do that instead of upgrading the environment to a newer version (let's say Node 8)?
There is a lot of real cases scenarios where that approach makes sense. One good example:
Let's say you have a legacy system that is running in an old environment, and you need to use such new methods to implement/fix a functionality. The upgrade of your environment still a work in progress because there are compatibility issues that require a lot of code changes and tests (a critical component).
In this example, you could try to craft your own version of such functionality, but that would make your code harder to read, more complex, can introduce new bugs and will require tons of additional tests just to cover a functionality that you know it will be available in the next release.
Instead, you can use this shim and make use of these new methods, taking advantage of the fact that this fix/functionality will be compatible after the upgrade, because you are already using the methods known to be available in the next specification. And there is a bonus reason: since these methods are native to the next language specification, there is a good chance that they will run faster than any implementation that you could have done if you tried to make your own version.
Another real scenario where such approach is welcome is at browser level. Let's say you need to support old browser and want to take advantage of these newer features. Javascript is a language that allows you to add/modify methods in its core objects (like adding methods to Array prototype), and those shim libraries are smart enough to add such methods only if the current implementation is lacking of them.
PS:
1) You will see the term "Polyfill" related to these Javascript shims. Polyfill is a more specialized type of shim that is used to provide forward compatibility in different browser level specifications. By the way, my example above refers exactly to such example.
2) Shims are not limited to this example (adding functionality that will be available in a future release). There are different use cases that would be considered to be a shim as well.
3) If you are curious about how this specific polyfill is implemented, you can open Javascript Array.find specs and scroll to the end of the page where you will find a canonical implementation for this method.
SHIM is another level of security check which is done for all the services, to protect upstream systems. SHIM Server validates every incoming request, with Headers User credentials, against the user credentials, which are passed in the request(SOAP / RESTFUL).