Is there a way to export or compile a KNIME workflow as a standalone Java application or JAR? I'd like to run the workflow on a platform where KNIME cannot be installed and/or as part of a larger program to simplify a complex but isolated piece of analytics. My options are many, but installing KNIME on the target platform is not one of them.
The only relevant reference I can find on the KNIME site is this ten-year-old(!) forum question. The only answer there links to this project which does seem to be active and says it is 'a KNIME extension that exports KNIME workflows to different workflow engines', though without digging into its code it's not clear what engines those are.
Other than that, I guess your options are:
ask on the KNIME forum again
since KNIME is open source and is based on Eclipse, look into the more general question of how to build and run a minimal standalone version of Eclipse - there seem to be some relevant-looking answers on here if you search, but I have no further knowledge on how to do it
use scripting nodes in KNIME to develop a text language version of your workflow, verifying as you go that the output adequately matches the KNIME nodes at each step, and deploy the text language version to your target. If you need data mining methods you might want to look at the Weka integration nodes which you could then substitute with calls to Weka methods.
Related
Problem started suddenly and seems to be not related to any change that happened in code around its first occurance.
We have automated CI workflow that prepares RC.
At first every second build worked. Now I had to rebuild RC 7 times to get app that is not crashing.
After investigating it is clear that there are no config values in resources, ergo plugin is not generating values from json config file. Basically rebuilding same code many times makes it work, at some point.
We had recently changed dependency injection in app from dagger components to hilt, it might be the reason of some kind of race or shared resources during build time.
It is also problematic to work around. I've prepared resource files with all values in order to manually add them in build process instead of generating with plugin. Now, because we also use crashlytics, I cannot remove google services plugin as crashlytics is checking for google services tasks in gradle. I cannot remove unused json file as plugin will crash because of lack of config file. Plugin also (based on docs https://developers.google.com/android/guides/google-services-plugin) adds some dependencies to main module, but I wasn't able to find anything in :app:dependencies so maybe docs are outdated.
I am not able to provide any project for reproduction as it only happens in commercial project. I understand it makes the problem hard to resolve in such case but maybe other people has it also.
AGP: 7.2.0-beta02
Google Services plugin: 4.3.10
Hilt: 2.41
EDIT:
I've resolved symptoms by creating gradle module for each build type we have, and each module contains only res/values.xml which I've copied from app/build/generated/res/google-services/*. This way required keys are always available in resources, as we do not often (hardly ever) change json file with config. I am still trying to investigate builds with --scan option but I cannot reproduce it with scan, it only happens without it.
Disclaimer: I am relatively unfamiliar with the flash build processes, so some/all of this may be misinformed nonsense. Please feel free to suggest alternative approaches.
We're currently developing a flex web app and our build situation is far from ideal. At present we're (as in individual developers) just building using FlashBuilder and deploying manually. The programmers are currently screaming bloody murder for two reasons, though:
The lack of CI is like going back to the stone age
We don't much care for FlashBuilder
(Note: We're only using FlashBuilder because it was the easiest way to set up a flex project in conjunction with Away3d and get it building / rendering correctly -- it's a stopgap solution).
As a predominately .NET development shop, we're used to doing continuous integration as well as continuous deployment. Ideally, we'd like to get something comparable to this for our flash projects without tying ourselves to a particular IDE.
Requirements:
The build process must be:
.. runnable via the commandline
.. runnable on both developer and CI build machines (and certainly not requiring an IDE!)
.. preferably as IDE-independent as possible (pragmatism will kick in though; if this causes a lot of friction we'll just pick one).
.. able to run on Windows (we develop using Windows)
We don't mind a touch of duplication or a few manual steps (e.g. tarting up the build scripts if we add a new project via an IDE, or generating one configuration from another if tools exist), but the less duplication / maintenance required the better.
I've read quite a few articles / blog posts and watched some short screencasts, but most of them are very thin on the ground on how the build system sits alongside IDEs. Most articles/screencasts have the same formula: How to create a "Hello World" build using a single file & text editors (no IDE).
I've not seen the topic of multiple libraries/projects etc. being broached, either.
After reading around the issue for a while, I'm considering investigating the following options:
Project Sprouts
Flexmojos
Maven Flex Plugin
buildr as3
Does anyone have any experience of the above solutions (or others I'm unaware of) and, if so, what do you make of them? Any help / pointers appreciated.
I recently started building with Gradle and the GradleFx plugin and I immediately fell in love with its power and ease of use.
Gradle is ANT + Maven + Ivy evolved and is primarily used from the command-line. You can:
write scripts in Groovy (a powerful Ruby-like language that runs in the Java Virtual Machine)
access all existing Maven and Ivy repositories as well as your own repos
use existing ANT tasks
integrate with CI (in Jenkins you just tick a checkbox to activate Gradle support)
although it has originally grown from the Java/Groovy community, it is in fact language agnostic. You add language-specific plugins for added functionality. GradleFx is such a plugin that provides you with additional ActionScript/Flex building tasks.
do easy multi-project builds. e.g. you can compile, unit test, package and deploy both your .NET service layer and your Flex client application with just one command.
use convention over configuration: if you stick to the conventions, your build scripts will be extremely terse
generate all kinds of reports: unit testing, checkstyles, codenarc, ...
generate Eclipse, IDEA or other IDE projects
all the things I haven't discovered yet
And best of all: it's very easy to learn. I had no knowledge of Maven before I started with Gradle and could get a multi-project build with some customizations working quite quickly.
Edit (comparison to Buildr AS3 and Maven)
I can compare this only to one of the projects you mentioned: Buildr AS3. It seems to start from a philosophy that is similar to Gradle's. But I've tried to use it about half a year ago and couldn't even get a simple 'Hello World' app to work. I e-mailed the developer for help: no response.
Compared to GradleFx: I had a small forum discussion with the developer (on a rather philosophical topic, since I didn't really need any help because it just worked right away). He answered within minutes.
As for Maven: (for what it's worth) I've only glanced at some configurations and they seem overly complicated when I compare them to a Gradle script.
There is one thing Maven does that you can't do with GradleFx (yet). It can pull the right Flex SDK from a Maven repo and build against that. With GradleFx you have to have your SDK's available locally.
I'm quite familiar with using maven as the main build tool and the flexmojos plugin from Sonatype. My experience has been a bit of a roller coaster with flexmojos. Maven is completely solid, it works all the time without issue, the only issue is the flexmojos plugin which has fluctuated a lot between versions. If you choose to go this route make sure to grab the source for flexmojos so you can see what your configuration options are actually doing to the command line parameters etc. For Flex 3.x flexmojos 3.x up to around 3.9 is good and works fine with regard to the goal for generating the .project eclipse files, believe there's also a mojo (a maven plugin) for generating intelliJ IDEA project files as well as others. If you're using Flex 4 you can compile with the latest flexmojos 4.0RC2 but it appears to me that the goal for generating flex/flashbuilder project properties is now gone (I'm not sure if this is because it's been replaced by another plugin altogether or what the deal is). However building with maven and flexmojos does fulfill all of your goals above (we also use it for building our service layer, so in a single mvn clean install we get a jar packed in a war packed in an ear with everything configured and a swf, that part is really nice). Also you can do continuous integration using bamboo (or simply write your own script that is triggered from a cron job or in windows as a batch file executed with a scheduled task if you don't have a *nix server around). Let me know if you'd like any more details or if I missed something major.
Shaun
I have been using Hudson, now Jenkins, with Ant for Flex automated builds and FlexUnit testing. Jenkins has some really useful plugins for integration with eclipse (and hence, FDT or FlashBuilder), Jira, SVN, Git etc., and it's free. Also, you can integrate the Ant build into Maven scripts, so I've found this to be a good and flexible solution for all purposes I've come across so far.
The Flex SDK comes with Ant tasks, and writing even elaborate Ant build scripts is quite easy - in fact, I'd been using Ant locally before, and I could reuse my existing scripts with only a few added extra compiler options for FlexUnit tasks.
However, it took a while to set up the system correctly for unit testing, because I'm running a headless server on Linux, and that implicates a rather complicated environment for ActionScript tests, because they run only in Flash Player. This, of course, is true for all CI scenarios using FlexUnit, regardless of which server you use.
Here's what I've learned:
FlexUnit needs a standalone debugger version of Flash Player installed, but Adobe only distributes binaries for the standard version on Linux. Therefore, compiling from source was necessary, and since my server system is stripped down to the bare necessities, it took some effort to install all the correct dependencies and get them to work.
The Flash Player needs hardware to run correctly: It uses graphics, therefore it needs a graphics card, and sounds, therefore it needs a sound card. On my headless server, this meant I had to install a VNC host to get it to run at all, and I had to eliminate any tests using sounds (those will now only run on local machines). If anyone ever comes across a working sound card emulation for openSuSE that I could use with the VNC client - you'll be my hero forever!
If you've set asynchronous timeouts in your unit tests, and/or you need to use setTimeout() to send delayed procedure calls, make sure the intervals aren't too short - I've had problems with tests that ran fine on any local machine, but broke the build on the CI server, because the Flash Player is considerably slower on the VNC client than on an actual graphics card.
I've also found this last issue to be a healthy lesson: Criteria for unit tests should not be based on assumptions about the system's performance, or at least be tolerant enough to succeed even on a slow machine.
Time and again I am faced with the issue of having multiple environments that must be configured individually for an application that would run in all of them (e.g. QA, regional production env's, dev, staging, etc.) and I am wondering what would be the best way to organize different configurations?
Would it be in the database? Different configuration files per environment? Or maybe the same file with different sections/xml tags? How would these be then deployed? Embedded within the app? Or put manually in after installation to be modified in-place?
This question is not technology-specific - I've worked with .net and Java, web-apps and desktop apps and this issue comes up time and again. I'm looking to learn different approaches to maybe adapt a hybrid to address this.
EDIT: There's one caveat that I must point out - when configuration is part of the deployed solution, it is generally installed under root user on the host. In large organizations developers usually don't have a root access to production hosts so any changes to the configuration require a new build and deployment. Needless to say this isn't the nicest approach - especially at organizations that have a very strict release process involving multiple teams and approval levels... (sigh I know!)
Borrowed from Jez Humble and David Farley's book "Continuous Delivery (page 41)", you can:
Your build scripts can pull configuration in and incorporate it into your binaries at build time.
Your packaging software can inject configuration at packaging time, such as when creating assemblies, ears, or gems.
Your deployment scripts or installers can fetch the necessary information or ask the user for it and pass it to your application at
deployment time as part of the installation process.
Your application itself can fetch configuration at startup time or run time.
It is considered bad practice by them to inject configuration files in build and compile times, because you should be able to deploy the same binary file to every environments.
My experience was that you could bake all configuration files for every environments (except sensitive information) to your deployment file (war, jar, zip, etc). And you design your application to take in an extra parameter when starts, to pickup the right sets of configuration files (from your extracted deployment file, or from local/remote file system if they are sensitive, or from a database) during application's startup time.
The question is difficult to answer because it's somewhat vague. There is no technology-agnostic approach to configuration as far as I know. Exactly how configuration is set up will depend on the language/technology in question.
I'm not familiar with .net but with java a popular approach is to have a maven build set up with different profiles. Each profile is specific to an environment. You can then define different properties files that have environment-specific values, an example from the above link is:
environment.properties - This is the default configuration and will be packaged in the artifact by default.
environment.test.properties - This is the variant for the test environment.
environment.prod.properties - This is basically the same as the test variant and will be used in the production environment.
You can then build your project as follows:
mvn -Pprod package
I have good news and bad news.
The good news is that Config4* (of which I am the maintainer) neatly addresses this issue with its support for adaptive configuration. Basically, this is the ability for a configuration file to adapt itself to the environment (including hostname, username, environment variables, and command-line options) in which it is running. Read Chapter 2 of the "Getting Started" manual for details. Don't worry: it is a short chapter.
The bad news is that, currently, Config4* implementations exist only for C++ and Java, so your .Net applications are out of luck. And even with C++ and Java applications, it won't make pragmatic sense to retrofit Config4* into an existing application. Because of this, I'd advise trying to use Config4* only in new applications.
Despite the bad news, I think it is worth your while to read the above-mentioned chapter of the Config4* documentation, because doing so may provide you with ideas that you can adapt to fit your needs.
I often find the quote "InstallUtil.exe" is an ugly pattern or "Don't use InstallUtil.exe" and that I should use native WIX or Installation package patterns and I still don't understood why.
I stepped away from using InstallUtil to install a .NET service as I finally learnt that writing registry keys for such an action should be an un-install-able action - and I've come to terms with this as correct.
As I've been working through my WIX installer for a relatively complex product, I have found myself in need of creating or updating SQL Server databases, creating or updating IIS Applications and finally updating or creating configuration files.
Each of my components (features) are optional, but they all share the same configuration file. As my product uses unity, its important to note that this library contains strong support for reading/updating/removing components from the Unity Configuration block, therefore it seems fairly smart to me that I should take advantages of these blocks via Installation Components (i.e. InstallUtil) to create or update my configuration file at installation time.
Just to be clear here, my installer does not natively contain a configuration file for my application: at installation time, the installer has no idea as to the shape of it as its based on the features selected. Surely I should be embedding this knowledge into each of the modules that are to be deployed and not in the remit of the installer which is now a completely independent project? Wouldn't this break O-O principals even if we are talking about installation?
I'd really appreciate some guidance as to whether this is good practise or not? Am I reading 'InstallUtil' is bad for installing services, or is it that using 'InstallUtil' is bad full-stop? If so, what are my options for smart updating of configuration files?
The main reason for avoiding InstallUtil is that it runs outside of the installation transaction, so Windows Installer cannot keep track of what it's done.
I have used InstallUtil on a few occasions, when I just couldn't get Wix to do what I needed and didn't have time to write a custom action. In this case I called the InstallUtilLib version as I feel this is a cleaner approach.
I used the this blog as a guide as to how to achieve this.
I'm developing an website now.
I found there are lots steps to do before deploying the code to web server.
e.g.
1. compact JS/CSS/HTML
2. Run unit test if any
3. Test code locally
4. Upload code to web server
Not sure if anyone could share your experiences on this process? Or how does facebook/google/yahoo ... do this?
Deployment strategy entirely depends on your project. None of those steps (except uploading code to web-server of course) is mandatory for running PHP web-site. Those steps appears when you are trying to:
Utilize build system (for configs generation, JS and CSS minification, etc);
Write unit-tests and (probably) run continuos integration;
Establish quality assurance by having QA go/no go before deployment of new release.
Needless to mention that to make, for instance, JS/CSS minification or perform unit-tests execution, you need to implement ones first.
That's why I'd not advice you to follow cargo-cult and copy all features of smbd's project (even facebook or google). Instead just follow the project needs and build your own deployment strategy.