I was going through some electron package.json examples where I found some interpolations like given below:
"updater": {
"urls": {
"darwin": "{{& SQUIRREL_UPDATES_URL }}/update/%CHANNEL%/darwin?version=%CURRENT_VERSION%",
"win32": "{{& SQUIRREL_UPDATES_URL }}/update/%CHANNEL%/win32",
"linux": "{{& SQUIRREL_UPDATES_URL }}/update/%CHANNEL%/linux"
}
}
"piwik": {
"serverUrl": "{{& PIWIK_SERVER_URL }}"
},
"sentry": {
"dsn": "{{& SENTRY_DSN_PRIVATE }}"
}
I do not really know the following:
what does this {{}} mean in json
where does these variable exist
what does & mean in {{}} "{{& SENTRY_DSN_PRIVATE }}"
If anyone can explain then it would be really kind. Many thank in advance.
I guess you are talking about Whatsie and it's package.json.
If you take a look at one of the Gulp tasks located in the file tasks/compile.coffee, you'll be able to see the lines (in CoffeeScript):
# Move package.json
gulp.task 'compile:' + dist + ':package', ['clean:build:' + dist], ->
gulp.src './src/package.json'
.pipe mustache process.env
.pipe gulp.dest dir
Here the actual package.json is being passed to a mustache template engine - it receives a template as a first argument (package.json here acts like a template) and a data to be inserted in the template as a second argument - process.env.
As package.json acts like a template for mustache, you can use mustache syntax in it.
Curly braces {{}} are the part of it, they are used as placeholders which will be replaced by the actual data, when templates are being compiled. In the mustache docs you can also find a line:
You can also use & to unescape a variable: {{& name}}
So {{& name}} is to prevent values from being escaped. Otherwise, if you don't use & and values for output have some dangerous characters , they will be replaced by more secure ones (originally to prevent XSS in templates), as a result it will transform initial value, which is not always what you want. In this case author wants to preserve original value.
Going back to process.env - it is an object which gives access to environment variables in Node.JS. There is a file in repository .env-example with an example of env variables developer has to set in order to have the application work differently in different environments (for example on local machine or CI server). Names of some of the variables in this file are the ones that are used in a package.json as template placeholders - I guess author of the app uses all of this to simplify a build process for different environments.
Related
I'm using SparkKubernetesOperator which has a template_field called application_file. Normally on giving this field a file's name, airflow reads that file and templates the jinja variable in it (just like script field in the BashOperator).
So this works and the file information is shown in the Rendered Template tab with the jinja variables replaced with the correct values.
start_streaming = SparkKubernetesOperator(
task_id='start_streaming',
namespace='spark',
application_file='user_profiles_streaming_dev.yaml',
...
dag=dag,
)
I want to use different files in the application_file field for different environments
So I used a jinja template in the field. But when I change the application_file with user_profiles_streaming_{{ var.value.env }}.yaml, the rendered output is just user_profiles_streaming_dev.yaml and not the file contents.
I know that recursive jinja variable replacement is not possible in airflow but I was wondering if there is any workaround for having different template files.
What I have tried -
I tried using a different operator and doing xcom push to read the file contents and sending it to SparkKubernetesOperator. While this was good for reading different files based on environment, it did not solve the issue of having the jinja variable replaced.
I also tried making a custom operator which inherits the SparkKubernetesOperator and has a template_field applicaton_file_name thinking that jinja replacement will take place 2 times, but this didn't work too.
I made an env file which had the environment details (dev/prod). Then I added this code to the start of my dag file
ENV = None
with open('/home/airflow/env', 'r') as env_file:
value = env_file.read()
if value == None or value == "":
raise Exception("ENV FILE NOT PRESENT")
ENV = value
and then accessed the environment in the code like this
submit_job = SparkKubernetesOperator(
task_id='submit_job',
namespace="spark",
application_file=f"adhoc_{ENV}.yaml",
do_xcom_push=True,
dag=dag,
)
This way I could have separate dev and prod files.
Facing issue while generating CF manifests through jinja2 templating. jinja2 is unable to output env property under single quotes in manifest.
The template.yml.j2 template file has this,
JBP_CONFIG_JAVA_MAIN:'{java_main_class: "com.test.example.GeneralService", arguments: "setup.yml"}'
The final manifest.yml has missing single quotes,
JBP_CONFIG_JAVA_MAIN: {java_main_class: "com.test.example.GeneralService", arguments: "setup.yml"}
Expected output in manifest,
'{java_main_class: "com.test.example.GeneralService", arguments: "setup.yml"}' with quotes
I am really not getting why single quote is getting omitted after rendering the output manifest.
With jinja, it's tricky but easy to do, try this in your template file -
JBP_CONFIG_JAVA_MAIN: "'{java_main_class: "com.test.example.GeneralService", arguments: "setup.yml"}'"
Works for me.
I want to include the value of the "version" parameter in package.json as part of the Jenkins build name.
I'm using the Jenkins Build Name Setter plugin - https://wiki.jenkins-ci.org/display/JENKINS/Build+Name+Setter+Plugin
So far I've tried to use PROPFILE syntax in the "Build name macro template" step:
${PROPFILE,file="./mainline/projectDirectory/package.json",property="\"version\""}
This successfully creates a build, but includes the quotes and comma surrounding the value of the version property in package.json, for example:
"0.0.1",
I want just the value inside returned, so it reads
0.0.1
How can I do this? Is there a different plugin that would work better for parsing package.json and getting it into the template, or should I resort to some sort of regex for removing the characters I don't want?
UPDATE:
I tried using token transforms based on reading the Token Macro Plugin documentation, but it's not working:
${PROPFILE%\"\,#\",file="./mainline/projectDirectory/package.json",property="\"version\""}
still just returns
However, using only one escaped character and only one of # or % works. No other combinations I tried work.
${PROPFILE%\,,file="./mainline/projectDirectory/package.json",property="\"version\""}
which returns "0.0.1" (comma removed)
${PROPFILE#\"%\"\,,file="./mainline/projectDirectory/package.json",property="\"version\""}
which returns "0.0.1", (no characters removed)
UPDATE:
Tried to use the new Jenkins Token Macro plugin's JSON macro with no luck.
Jenkins Build Name Setter set to update the build name with Macro:
${JSON,file="./mainline/pathToFiles/package.json",path="version"}-${P4_CHANGELIST}
Jenkins build logs for this job show:
10:57:55 Evaluated macro: 'Error processing tokens: Error while parsing action 'Text/ZeroOrMore/FirstOf/Token/DelimitedToken/DelimitedToken_Action3' at input position (line 1, pos 74):
10:57:55 ${JSON,file="./mainline/pathToFiles/package.json",path="version"}-334319
10:57:55 ^
10:57:55
10:57:55 java.io.IOException: Unable to serialize org.jenkinsci.plugins.tokenmacro.impl.JsonFileMacro$ReadJSON#2707de37'
I implemented a new macro JSON, which takes a file and a path (which is the key hierarchy in the JSON for the value you want) in token-macro-2.1. You can only use a single transform per macro usage.
Try the token transformations # and % (see Token-Makro-Plugin):
${PROPFILE#"%",file="./mainline/projectDirectory/package.json",property="\"version\""}
(This will only help if you are using pipelines. But for what it's worth,..)
What works for me is a combination of readJSON from the Pipeline Utility Steps plugin and directly setting currentBuild.displayName, thusly:
script {
// readJSON from "Pipeline Utility Steps"
def packageJson = readJSON file: 'package.json'
def version = packageJson.version
echo "Setting build version: ${packageJson.version}"
currentBuild.displayName = env.BUILD_NUMBER + " - " + packageJson.version
// currentBuild.description = "other cool stuff"
}
Omitting error handling etc obvs.
UPDATE -- See additional info below, added on 22 March...
I want to inject an SCSS variable into a code file with my Gulp process.
I think this is two steps:
Gulp the SCSS _variables.scss file so I can access the variable
Just do a simple gulp.replace to do a string-replace and drop the
variable value into my file (like the answer here)
I'm stuck on step 1. Here's what I have... as best I can tell, the not-heavily-documented gulp-sass-variables plugin is supposed to pull the variables from the SCSS file into the argv object. So I followed the example on that plugin page, and tried this:
gulp.task('test', function () {
gulp.src('sass/*')
.pipe(plugins.print())
.pipe(plugins.sassVariables({
$foundIt: argv.testSassVar
}))
console.log(argv);
});
The print() call is just to confirm that it is reading my SCSS files (and it is). But it can't find $testSassVar in my SCSS code, as you can see from this error message:
[12:53:04] Using gulpfile ~/dev/project/gulpfile.js
[12:53:04] Starting 'test'...
[12:53:04] 'test' errored after 15 ms
[12:53:04] ReferenceError: testSassVar is not defined
at Gulp.<anonymous> (...)
[12:53:04] sass/_mixins.scss <- this line and all below are from print()
[12:53:04] sass/_normalize.scss
[12:53:04] sass/_variables.scss
[12:53:04] sass/footer
[12:53:04] sass/header
[12:53:04] sass/landing
[12:53:04] sass/landing-style.scss
[12:53:04] sass/site
[12:53:04] sass/style.scss
I tried removing that $foundIt assignment, and just trying to fill the argv object, but that ends up outputting an empty array:
[13:00:57] Using gulpfile ~/dev/project/gulpfile.js
[13:00:57] Starting 'test'...
{ _: [ 'test' ], '$0': 'gulp' }
[13:00:57] Finished 'test' after 41 ms
Any ideas how I can accomplish what I'm trying to do?!
EDIT 22 March
Looks like I can partially accomplish this by going a different route and using the gulp-sass-json plugin:
gulp.task('sass-json', function () {
return gulp
.src(scss_base + '*')
.pipe(plugins.sassJson())
.pipe(gulp.dest(tmp + 'sass-vars.json'));
});
This writes a file to /tmp with my SCSS variables compiled into a JSON file. Then I thought, ok, I can load the JSON file in gulp and parse it from there. Unfortunately, this plugin doesn't resolve 2nd level variables (i.e. variables that reference variables), so you get something like this:
{
# variables parsed from SCSS to JSON by gulp-sass-json
# these are ok
"font__main": "'Open Sans', sans-serif",
"font__heading": "'Open Sans', sans-serif",
"font__line-height-body": "1.5",
"testSassVar": "123",
"color__nbar": "#ffffff",
"color__nbar_bg": "#50B107",
"size_border-radius": "3px"
# these variables reference other variables, not parsed, ouch
"color__background-body": "$color__nbar",
"color__background-screen": "$color__nbar",
# and this is even worse, the SCSS function isn't parsed, double ouch
"color_test": "lighten($color__nbar, 50%)",
}
As you can see, the variables that are simple references to other variables could probably be fixed with a bit of Javascript to just parse them, but once you get into the SASS functions, that idea falls apart.
Ideally, the SCSS compiler could parse and variables that aren't static values. Is that possible (or is that starting to really get into the weeds to solve this problem)?
The only other thought I had was to move all my variables to a JSON file, then use a gulp plugin that moved in the other direction -- JSON to SASS variables. But then I wouldn't get the advantage of having the SCSS compiler to run functions, parse variables, etc, and then give those values back to my gulp file.
Since my goal is to have the SASS values in the (parsed) variables available to my gulp file, so I can turn around and drop them into another (HTML or PHP) code file doing a simple string replace. If there's an easier way to accomplish my final goal, I'm all ears...
I recently had a similar issue, although I'm not using gulp.
I searched for npm packages that could accomplish this task (variable extraction from SCSS), but the couple I found were subpar.
So, I ended up writing my own lib (extract-scss-variables).
We use it in production, it's tested against foundation and a couple of other common uses.
I sincerely don't remember how to produce a gulp plugin anymore, but this library in itself shouldn't be hard to integrate.
In your specific case I'd do something like:
var extractScssVariables = require('extract-scss-variables');
var variables = extractScssVariables({
entryPoint: 'path/to/sass/style.scss',
files: [
'path/to/sass/_mixins.scss',
'path/to/sass/_normalize.scss',
'path/to/sass/_variables.scss',
],
sassOptions: {
includePaths: ['path/to/libraries/to/include.scss'],
},
});
where the entryPoint is the context where the variables are extracted (e.g. if you overwrite a variable declared in _mixins.scss, the entryPoint's one is used instead);
files is the array of files where to extract SCSS variables from;
and sassOptions are any command line sass options you are already using.
I've also authored a library for this kind of use case that uses native sass compiler feature to extract the variable values called sass-extract which also comes with a webpack loader and babel plugin depending on the use case.
The benefit is that is based on the native sass type system and thus supports all kinds of sass features like functions, mixins, lists, maps etc. It also handles imports, overrides, defaults etc as expected.
If you are open to using webpack you can simply import the sass file:
const style = require('sass-extract-loader!./style.scss');
to get the variables extracted, which would be the easiest option. Otherwise you can just use the core library directly as such:
const sassExtract = require('sass-extract');
sassExtract.render({
file: 'path/to/my/styles.scss'
})
.then(rendered => {
console.log(rendered.vars);
});
There is also a synchronous version available that you can use to integrate with your gulp pipelines.
Here is my solution - you need to have gulp-inject
main.scss file
...
/* inject:scss */
/* endinject */
...
gulpfile.js file
var gulpInject = require('gulp-inject');
...
gulp.src('...your/main/scss/file.scss')
.pipe(
gulpInject(gulp.src('...your/partial/scss/**/*.scss'), {
relative: true,
transform: function (filePath) {
var fileWithoutExtension = filePath.replace('.scss', '');
return '#import \'' + fileWithoutExtension + '\';';
}
})
)
.pipe(...)
...
The above example will inject partial SCSS files into the main SCSS file.
Hopefully my code would be helpful.
Here is a variable that is defined in my local .env file in my app. I created it to be a comma separated value, like so:
STATE_KEYWORDS=georgia,new york,new jersey,maine,vermont,florida
In my seed.rb file, I call on that STATE_KEYWORDS variable by using the "fetch" and "split" methods to turn it into an array, because I need that attribute ("keywords") to be an array:
Category.create(name: "U.S. States", keywords: ENV.fetch("STATES_KEYWORDS").split(","))
This works fine when I run my app remotely, but when I push to Heroku, it seems as if Heroku is not recognizing/picking up the .env file. So I tried to add the variable like this:
heroku config:add STATE_KEYWORDS=georgia,new york,new jersey,maine,vermont,florida
But then I get this error:
new york,new jersey,maine,vermont,florida
is invalid. Must be in the format FOO=bar.
I don't think it likes the space between "new" and "york". Or "new" and "jersey"...etc. It wants it to be one fluid value.
But I need the "keywords" array to equal the below, with some elements in that array being 2 or 3-worded strings:
keywords: ["georgia", "new york", "new jersey", "maine", "vermont", "florida"]
How can I do this? How can I add this .env variable to Heroku (i'm not married to leaving them in an .env file, I just can't have them pushed up to my Github/Heroku in plain view, for everyone to see. Those keywords have to be hidden...which is why locally I had them in a .env file.
Also, I have ".env" in my.gitignore file, which is why I successfully pushed to Github without that file appearing there.
Remove Spaces when you are commanding!
like
incorrect heroku config:set key= value
correct heroku config:set key=value
Try this:
STATE_KEYWORDS='georgia,new york,new jersey,maine,vermont,florida'
Remove any spaces in your .env file