How do you manage external dependencies for your application? - language-agnostic

There are many types of external dependencies. Interfacing with external applications, components or services (e.g. Outlook to send emails, TWAIN or WIA for scanning, ActiveX objects and Web services for various purposes, and so on).
What is your strategy for making sure that your application is robust, and can run even when such external dependencies (for innumerable reasons) are unavailable?
Note: Code from external sources included in your own source code is another type of external dependency, but that is not mainly what I am concerned about here.

Our strategy, is that our software may not except in exceptional circumstances. So we program quite defensively:
If there is no mail connection, the mail functionality is not available.
Graphic output depends on available graphics card.
Only the availability of a network enables the distributed functionality.
And there are more.

Also note, that with external interfaces, much as with user input, you should not trust that input. Always validate input, and check for consistency (or existence) when calling.

Related

Can I make calls to APIs such as youtube-dl and ffmpeg from a chrome-app?

First of all, I haven't started the implementation of the system I'm about to describe, as I didn't want to commit on implementing something I did not know if was possible.
So, what I'm trying to achieve is to build a chrome-app to download the audio from certain websites (e.g. youtube and soundcloud) using youtube-dl, post process it using ffmpeg and then upload it to a cloud service via some api. The reason I want to do it via a chrome-app is because I could do all the work on the client side (no need for servers) and I'd have the ability to insert javascript into the pages using content scripts, which would make the app pretty simple to use (I could create buttons such as 'download song' and stuff like that).
Although I have already read the documentation explaining the NaCl Technical Overview and some of the Application Structure, I still am not sure as to whether I would be able to make these calls via some C/C++ module or if I would get denied due to security reasons.
To summarize: considering that the user has the needed dependencies in his system (youtube-dl, python, ffmpeg and etc.), is it possible to make calls to third party APIs such as the ones described before via a chrome-app using NaCl ?
Thank you all in advance,
Chrome apps are normally sandboxed.
Less so than extensions - they can reach much more system resources via app APIs.
But still, what you mention is executing libraries / utilities out of browser, and it's not normally allowed.
(P)NaCl is tightly sandboxed in this regard. See this old question, it still applies: you can only use 3rd-party code that compiles into NaCl along with your app, not just link to a library. There are some library ports to NaCl, but it's not automatic.
Normally, a few years back you would use a mechanism like NPAPI to reach out and use a library out of browser. It's deprecated, and won't work anymore. In its place, Chrome offers a pipe-like (through stdio) connection to an external program called Native Messaging. You could use it to perform operations with system-level libraries and tools, but the downside is that you can't bundle the native host with your app, you'll need a separate installer.

JSP / HTML UI Design

We have some JSP code to run in our pages.
There is a UI designer who will constantly update the UI but does not have Tomcat server. We prefer he doesn't because he isn't a programmer.
However, it is getting annoying to have to cut and paste the JSP related code each time he updates the UI.
Is there anyway to handle this issue? We prefer to keep the same files, but still have it so he can see his UI work without worrying about the JSP and when he checks in the new files, we don't have to cut and paste our JSP related code.
One example of such code, is that there are certain navigation menu items which are displayed depending on the user.
We are using Tomcat authentication. We could I suppose use AJAX to obtain the user information, but is that less secure? Everything else in the application is AJAX.
The problem here is that this person is not working with the team. Rather, he's creating work for them... and it goes both ways. Read on.
I both do and manage front end development. If this UI person was on my team, I would force him to set up a Tomcat server. He just needs to learn some things.
In effect, when implemented properly, JSP is not much different from any other server-side markup language for views, such as Rails + ERB, PHP, .NET etc... even Javascript templating engines (mustache, handlebars, etc.). The same condition checks, for-loops, auth checks - all basic view-layer logic that is needed is available and usable.
If he's on a Java project / team, he needs to learn the Java front-end. It's that simple.
His main tasks should be basic, and frankly, he shouldn't even need to install a Java IDE to do them. They are:
Get/push source code + analyze diffs (any source control client)
Build / deploy latest to his local environment (scripts or .bat files)
Work on the running app*
(*) The last part is where things get tricky. If you work directly on the running server and then accidentally run a fresh deploy before copying over your updates, you're screwed. If you use symlinks (which are also available in Windows), there may be files that only appear post-compilation, or locks, or sync issues when getting latest code - all creating problems.
The way I have found that works best is to work on the code repo location (pre-build), and create two scripts:
Build+deploy - stops running server, blows out directories and caches, builds latest, and redploys
Update - Synchronizes the View files and any other necessary directories with the deployment target. You must be sure to disable hot-deploy in the Tomcat config, or you'll get memory leak errors.
That said, and it should be obvious by now: Java is one of the most difficult ecosystems to develop UI's for. The compiled nature and complex environment requirements make development slow and tedious, with significant dependencies on different people or systems to make a decent product.
JSP itself, while capable as described above, is almost always organized badly, with various ways of includes, tagfiles, partials, frameworks - it becomes a UI person's worst nightmare. GSP (from Grails) solves a lot of the organizational issues, but will require flexibility from the dev team. Even then it is not an "ideal" solution.
JSP syntax - JSTL, C:tags, etc. creates even greater headaches. Front-end people who do not program, don't use IDE's and therefore don't have a way of looking up methods, objects, parameters etc. when writing or customizing conditional logic or loops. The dev team can help by pre-writing these out on the page, but any time there are changes or enhancements needed, it requires meetings, conversations, and compromise.
In the long run, you should abstract the Java app from a separate, more flexible, more capable front-end technology stack, using REST / JSON-based services to talk between the two. (Side note: For performance / apps with scale, ensure you are using either a custom protocol or Web Sockets).
My preference is node.js, because front-end developers can stick with the language they will know best: Javascript / JSON. But it could be anything that your particular front-enders are comfortable with and can do design with.
The key is to eliminate bottlenecks on both front-end and back end. Both tracks should be able to develop and iterate quickly, with the RESTFUL API being the key point of collaboration.
Lastly, for those of you who are aspiring front-end developers / designers but only know Java (or some other server-side technology), I CHALLENGE YOU to learn something new. User-facing technologies are in a constant state of change, and more recently that change has accelerated. If you want to have UI-competitive products, you need to invest in technologies that will make them competitive.

Delphi with HTML/CSS interface

I want to develop a delphi application with an HTML/CSS graphical user interface, not necessarily running from within a web browser. I want to do this do create a richer graphical user interface with animations etc. and break away from normal VCL components / Windows look. Any suggestions?
HTML and CSS won't deliver animations or a Rich User Interface to your feet. Far from it in fact. Quite the opposite. You will need to invest in a toolkit to provide that sort of functionality and almost certainly involve JavaScript. And even if you don't want your eventual application hosted in a web browser, your application will itself have to host a web browser to render your HTML/CSS/JavaScript UI, and you will then have a much more difficult job of connecting your GUI to your application logic (unless you do actually embrace a web application architecture).
Delphi (or any Windows application development language for that matter) gets you much, much further down the road towards a more simply, effectively and quickly implemented Rich User Interface than HTML or CSS.
If you don't like the look and feel of the standard Delphi controls (which in essence is what you are saying) there are numerous alternative libraries available.
Also bear in mind however that when someone uses a Windows application they expect it to look and behave a certain way to a large extent. Using fancy, web based paradigm's in a desktop application simply for the sake of it is likely to confuse and frustrate users if taken too far.
I'm all for user interfaces breaking with convention where it leads to a more intuitive user interface, but simply being "prettier" does not necessarily lead in that direction and is just as misguided as dogmatically adhering to convention.
In one of my applications I have an an embedded browser and I have implemented the IDocHostUIHandler interface. This allows me to expose a COM object via the "GetExternal" method. I simply have a COM object that exposes methods and properties of my application which makes them available to the web pages hosted inside the embedded browser.
So the script in my web pages has lines like "external.DoSomething()" and "i=external.GetThisValue()". So, for example, behind button onclick events you can run a method of your application (implement in the main form, in the COM object itself, or whatever you like).
This site has lots of info on embedding a browser in your Delphi app:
http://www.delphidabbler.com/articles?article=22
It can certainly be cumbersome to implement a lot of this stuff and in many cases there are probably better options. But for my specific purpose I am able to offer a "home page" which can easily be modified to change its layout, look and even expose more (or less) functionality as required by myself or my users.
If you want a Delphi program with a better-looking interface, HTML is really not what you're looking for. What you really need are better-looking VCL controls.
Take a look at TMS Smooth Controls, for example. If you're on Delphi 2009 or 2010, you can get it as a free download here. That's one of many component libraries that can bring a slicker user interface to your program.
HTML / CSS offers some nice features which are (not yet?) available in Delphi and the VCL. They are also a good starting point for client/server programming, separating the user interface and the business logic is a key factor here.
One popular library for Delphi is the extpascal project:
ExtPascal is an Object Pascal (Delphi,
FreePascal/Lazarus) wrapper/binding
for Ext JS, a complete GUI Ajax
framework, made in JavaScript, for
Rich Internet Application (RIA)
development. ExtPascal lets you use
Ext JS from Object Pascal commands
issued by the server. It brings the
structure and strict syntax of the
Object Pascal for programming the web
browser. ExtPascal will wrap Draw2d
into future releases.
Some demos are online here and here.
p.s. and I really like the HTML / CSS support for element and font sizes in relative units (for exampe percent). Combined with browser zoom in / zoom out and WCAG, user interface ergonomy can not be much better.
HTML Option 1
If you relly want to use HTML+CSS(+JavaScript) to build a GUI, you can have a look at HTML Applications, a very fascinating concept from Microsoft. HTML Applications, .hta files, have been supported from Windows ME, if I remember correctly, and they are still supported on Windows 7.
You could create an HTML Application (i.e. an HTA file), and by so doing, creating a GUI using only HTML, CSS, and JavaScript. When the user double-clicks the HTA file, it will open like a program, but the GUI is entirely based on HTML; in fact, the entire Window is an Internet Explorer window in disguise.
And now comes the important part: you could create non-GUI Delphi applications (i.e., Delphi applications that are not console applications, but that have no forms either), and start them via hyperlinks (or JavaScript) from within your HTA GUI. (Well, it is probably better to create one such Delphi application, and use command-line arguments (ParamStrs) to communicate the desired action.)
Just an idea...
HTML Option 2
Alternatively, you could create a normal Delphi GUI application, but fill the entire main form with a TWebBrowser (a IE control), using Align := alClient. Then you could either load static HTML pages (stored in the Program Files folder or on the Internet), or you could use Delphi to dynamically create HTML pages to show. I think it is possible to intercept links from the control, so that you could respond to links using Delphi code.
What about OpenGL?
If you want to "break away" from the normal Windows look and feel, then I would recommend you to create your GUI using OpenGL. It is very easy to make a Delphi application with OpenGL (as long as you are familiar with OpenGL) - just add "OpenGL" to your uses list.
First this: I completely agree with Deltics' answer.
Having said that, if you master HTML and CSS (and JavaScript and AJAX etc etc) and you are looking for a way to use the power and speed of the Delphi compiler to run the dynamics of a website, this may be of interest.
I've created a project that uses the Delphi compiler to build a library that runs a website. The source-files combines HTML and Delphi, much like other web-scripting tools out there, but gets processed on a page-refresh, and compiled automatically. It uses a 'library handler' that plugs in the website library into pretty much anything you like: IIS, Apache, a stand-alone HTTP server (for hosting), or directly into InternetExplorer or FireFox (which is great for developing).
http://xxm.sourceforge.net/
New versions of Qt contain ability to use html/js for interfaces. I don't know if there is Qt library bindings for Delphi, but Qt is exactly what you want.
for Rich GUI and animation, have you looked at KSDev DXScen and VGScene ?
If you want to keep your delphi/Pascal Object 'background' and have a Web like RIA you also have a look to Morfik : link text

How to design extensible software (plugin architecture)? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I need some resources that talk about how to design your software to be extensible, i.e. so that other people can write add-ons/plug-ins that adds functionality to it.
What do you recommend? Any books out there that discuss the subject?
I would prefer something that's short and to the point; a bit of theory and a bunch of concrete examples.
I'm not targeting a specific language, I want to be able to understand the core idea so that I can implement it in any language.
And for the same reason, I prefer not to do it using a framework that someone else built (unless the framework is not very high-level, i.e. doesn't hide too much), at the moment I only want to educate myself on the subject and experiment with various ways to implement it. Plus, a framework usually assumes user's knowledge about the subject.
UPDATE
I'm not asking about OOP or allowing my classes to be inherited. I'm talking about designing an application that will be deployed on a system, such that it can be extended by third-party add-ons AFTER its been deployed.
For example, Notepad++ has a plug-in architecture where you can place a .dll file in the plugins folder, and it adds functionality to the application that wasn't there, such as color-picking, or snippet insertion, or many other things (a wide range of functionality).
IF we're talking .NET, try Scripting .NET applications with VBScript over on CodeProject. Lots of concrete examples there.
Below are sites implementing various application extension techniques
ClearScript - Makes V8, VBScript and JScript available to .NET apps
CS-Script - The C# Script Engine
Plugin Architecture using C#
Opinio plugin architecture
Notes on the Eclipse Plug-in Architecture
Plug-in Architecture Framework for Beginners
Gecko plugin architecture
Fungimol plugin architecture
OSGI is a good practical example of a technical framework allowing to do what you are after.
The theory is here.
The (free!) book is there.
Extensibility and the ability to write plugin must deal with service lifecycle
adding / removing service/plugin on the spot
managing dependencies between services
managing states of services (declared, installed, started, stopped,...)
What is OSGI for ?
One of the main functions of a module is as a unit of deployment… something that we can either build or download and install to extend the functionality of our application.
You will find a good introduction here, on the central notion of service (which is related to your question, and which explain some problems around services, key component for extensibility).
Extract:
Why are services then so important if so many applications can be built without them? Well, services are the best known way to decouple software components from each other.
One of the most important aspects of services is that they significantly minimize class loading problems because they work with instances of objects, not with class names. Instances that are created by the provider, not the consumer. The reduction of the complexity is quite surprising
Not only do services minimize configuration, they also significantly reduce the number of shared packages.
Implement SOLID principles in your application.
1. Single responsibility principle: A class should have only a single responsibility (i.e. only one potential change in the software's specification should be able to affect the specification of the class
2.Open/closed principle: Software entities … should be open for extension, but closed for modification
3. Liskov substitution principle: Objects in a program should be replaceable with instances of their subtypes without altering the correctness of that program
4. Interface segregation principle: Many client-specific interfaces are better than one general-purpose interface
5. Dependency inversion principle: One should Depend upon Abstractions. Do not depend upon concretions
Stackoverflow questions:
Example of Single Responsibility Principle
Is the Open/Closed Principle a good idea?
What is the Liskov Substitution Principle?
Interface Segregation Principle- Program to an interface
What is the Dependency Inversion Principle and why is it important?
You try to reach two competing goals:
The components of your software must expose a lot of themselves, so they can be reused
The components of your software must expose very little of themselves, so they can be reused
Explanation: To encourage code reuse, you should be able to extend existing classes and call their methods. This isn't possible when the methods are declared "private" and the classes are "final" (and can't be extended). So to meet this goal, everything should be public and accessible. No private data or methods.
When you release the second version of your software, you will find that many of the ideas of version 1 were plain wrong. You need to change many interfaces or your code, method names, delete methods, break the API. If you do this, many people will turn away. So in order to be able to evolve your software, the components must not expose anything that is not absolutely necessary - at the cost of code reuse.
Example: I wanted to observe the position of the cursor (caret) in an SWT StyledText. The caret is not meant to be extended. If you do it, you'll find that the code contains checks like "is this class in the package org.eclipse.swt" and a lot of methods are private and final and whatnot. I had to copy about 28 classes out of SWT into my project just to implement this feature because everything is locked down.
SWT is a nice framework to use and hell to extend.
Of course there is the famous Open Closed Principle - http://en.wikipedia.org/wiki/Open/closed_principle
Well it depends on the language.
In C/C++ I'm pretty sure there is a loadlibrary function that allows you to open a library at runtime and invoke it's exported functions. This is typically how it's done in C/C++.
In .NET, there is Reflection, which is offers similar (but more broad) to loadlibrary. There is also entire libraries built on Reflection like Managed Extension Framework, or Mono.Addins that does most of the heavy lifting for you already.
In Java, there is also Reflection. And there is the JPF (Java Plugin Framework) which is used in stuff like Eclipse IIRC.
Depending on what language you use I could recommend some tutorial/books. I hope this was helpful.
Plugin architecture is becoming very popular for its extensibility and thus flexibility.
For c++, Apache httpd server is actually plugin based, but a concept of module is used instead. Most of apache features are implemented as modules, like cache, rewrite, load balancing, and even threading model. It is a very modular software I ever saw.
And for java, Eclipse is definitely plugin based. The core of Eclipse is an OSGI module system which manage bundles, another concept for plugin. Bundle can provide extension points on which we can build modules with less efforts. The most intricate thing in OSGI is its dynamic characteristic, which means bundles can be installed or uninstalled at runtime. No stop-the-world syndrome any more!
Since I dont have enough rep points to leave a comment, I am posting this as an answer. SharpDevelop is an IDE for developing applications in C#/VB.NET/Boo. It has a pretty impressive architecture that allows itself to be extended in a number of ways - right from new menu items to development support for whole new languages.
It uses a bit of XML configuration to act as a glue layer between a core of the IDE and the plugin implementation. It handles locating, loading and versioning of plugins out of the box. Deploying new plugins is matter of simply copying in the new xml configuration file and the required assemblies (DLLs) and restarting the application. You can read more on this in the book "Dissecting a csharp application" by the original author(s) - Christian Holm, Mike Krüger, Bernhard Spuida of the application from here. The book doesnt seem to be available on that site, but i found a copy that might still be around here
Also found a related question here
Checkout "CAB" - Microsoft's Composition Application Building blocks Framework. I think they've got a "web version" of that too...
I have just started to develop a smart client application. These are two options I am considering.
Using Microsoft's System.AddIn namespace. Looks very promising, however it may be a little complex for our end solution.
Or the Smart Client - Composite UI Application Block from Microsoft
Recently, i have looked at taking components both the Composite UI Application Block and the System.AddIn namespace to build my own. Since source code is available for the CAB it is easy to extend. I think our end solution will be a light weight version of the CAB, definatly using the Unity Application Block
If you work with .Net, our research yielded two approaches: scripting and composition.
Scripting
You extend the functionality of what your classes can do by orchestrating them using scripts. That means exposing what is compiled in your favorite .Net language in a dynamic language.
Some options we found worth exploring:
IronPython
IronRuby
JavaScript: Jint, Jurassic and JavaScript .Net are good starting points.
Script.Net -> this one was the first one to call our attention.
Composition
If you start a project with .Net 4 or above, you must take a good look at the Managed Extensibility Framework (MEF). It allows you to extend the functionality of your apps in a plugin way.
The Managed Extensibility Framework (MEF) is a composition layer for
.NET that improves the flexibility, maintainability and testability of
large applications. MEF can be used for third-party plugin
extensibility, or it can bring the benefits of a loosely-coupled
plugin-like architecture to regular applications.
Managed Add-in Framework is also a good read.
MSDN: http://msdn.microsoft.com/en-us/library/dd460648.aspx
Codeplex: http://mef.codeplex.com/
Rather than re-inventing the wheel, use the frameworks in hand. Eclipse and Netbeans both support plugin based extensions. You have to work in Java though.

MS CRM Development Projects

The shop that I am working part time with is new to Microsoft CRM. I just want to survey what projects have developers done to the system to extend its capabilities.
I can break the work I did into four sections:
Tailoring - Simple field level changes. A lot of this is just making sure the fields and language suited the business I was developing for.
Customisation - More complex changes, generally needing JavaScript and maybe ASP.NET. Some examples would be to use an IFrame and pass values to it from a CRM form. The IFrame would then do interesting things like mapping, charting or give you buttons to do other things. For buttons I would often times use JavaScript to replace the outerHTML in the HTML dom of an IFRAME to show a button rather.
Integration - using .NET to connect MSCRM to other systems. Connected it to Great Plains, Speech Server, SCOM (was called MOM back then), custom LOB systems etc... One interesting one I did was to develop a SSIS component that wrote into MSCRM via the web services.
Reporting - Building reports. In reporting services and in Excel. Excel made for great dashboards because of the dynamic update nature of it.
I have a few (war) stories up at http://www.sadev.co.za/taxonomy/term/7 if you are interested.
I (and others) have implemented a LINQ query provider for the web service layer http://www.codeplex.com/LinqtoCRM.
I did some work with CRM 3.0. My work enhanced the program and turned it into a Document Management app, where you could scan and upload documents based on a case, contact, customer, vendor etc. The .NET SDK back then could have used a bit more work, but I hear with newer versions of CRM it has gotten better. CRM allows for attachments but not at all levels, more at the case level.
We extended 3.0 to send text messages rather than email. There are tools available (for 4.0 at least) to do that now, though.
We also extended it to push service appointments through to Exchange, as due to the workforce being mostly mobile they did not have the CRM client for Outlook.
Two other major areas of development not yet mentioned are plugins and custom workflow assemblies.
Plugins allow you to intercept many of the messages that CRM processes to insert custom logic. The possibilities this gives you are endless because you can intercept all kinds of messages including Create, Update, Delete, and even Read!
Custom workflow assemblies are assemblies that you register with CRM that integrate with CRM's workflow UI. Often times, they perform simple tasks that are useful when modifying form data. Other times, they are more complex like sending status reports.
Custom workflow assemblies are always listed beneath the native options. In the screenshot, everything below "Stop Workflow" was added via a custom workflow assembly.