I am looking to filter out specific parts of a json file so that the given part of the file does not get pulled into a git repository. My use case is that I am setting up a repository to keep some working files, including settings for vsCode. I have a plugin for window colors that sets different colors for different windows that are open. The current color is saved in the .vscode/setting.json file for that window.
I found where it is possible to use the .gitattributes file to apply a filter to a file or set of files, and then use "$git config" to remove certain lines from what is committed, based on a sed command per this previous question.
I would like to apply this to the "workbench.colorCustomizations" object within the following json file, so that this object does not get committed, while other settings in the file may be committed, such as the "editor.formatOnPaste" object. Does anyone know of a way to do this?
{
"workbench.colorCustomizations": {
"activityBar.background": "#102D56",
"titleBar.activeBackground": "#173F79",
"titleBar.activeForeground": "#F8FAFE"
},
"editor.formatOnPaste": true
}
Related
I am new to opa/rego and have an opa test that I would like to run. Within that test .rego file, I would like to use an input.json file from a different directory. Is there a way to specify that file within a "with input as _____" statement within the test file? i.e.
test_allow {
allow with input as <path-to-file>
}
My thoughts so far have lead me to trying the -b option but the directories are pretty far apart and I do not want a bundle that large and dependent. Additionally, I have thought about the import statements but I do not have the "-i" option within the opa test subcommand. I've also tried specifying each file (the .rego policy file, the .rego test file, and an input file) within the opa test subcommand to no avail.
Any help is greatly appreciated
OPA, and by extension the OPA test runner, doesn't really consider files at the time of policy evaluation. When OPA starts, all the files/directories pointed out by the command are merged under the data attribute, and may then be referenced byt their path, e.g. data.my-mocks.mock1, and so on.
If you want to include "test data" in your tests, you could keep those files in a directory included when running the opa test command. Since all data files are merged though, you'll need to ensure there aren't conflicting paths in those files. This is commonly accomplished by using a unique "top level" attribute per item. Something like:
{
"mock1": {
"item": "here"
},
"mock2": {
"item": "here"
}
}
You may then reference this in your tests like you suggested:
test_allow {
allow with input as data.mock1.item
}
I am trying to do some optimization in ADF. Setup is a third-party tool copies one JSON file per object to a BLOB storage container. These feed to a Mapping Data Flow. The individual files written by the third party tool work great. If I copy these files to a different BLOB folder using an Azure Copy Data activity, the MDF can no longer parse the files and gives an error: "JSON parsing error, unsupported encoding or multiline." I started this with a Merge Files, but outcome is same regardless of copy behavior I choose.
2ND EDIT: After another day's work, I have found that the Copy Activity Merge File from JSON to JSON definitely adds an EOL character to each single JSON object as it gets imported to the Merge file. I have also found that the MDF fails definitely with those EOL characters in the Merge file. If I remove all EOL characters from the Merge file, the same MDF will work. For me, this is a bug. The copy activity is adding a character that breaks the MDF. There seems to be a second issue in some of my data that doesn't fail as an individual file but does when concatenated that breaks the MDF when I try to pull all the files together, but I have tested the basic behavior on 1-5000 files and been able to repeat the fail/success tests.
I took the original file, and the copied file, ran them through all of sorts of test, what I eventually found when I dump into Notepad++:
Copied file:
{"CustomerMasterData":{"Customer":[{"ID":"123456","name":"Customer Name",}]}}\r\n
Original file:
{"CustomerMasterData":{"Customer":[{"ID":"123456","name":"Customer Name",}]}}\n
If I change the copied file from ending with \r\n to \n, the MDF can read the file again. What is going on here? And how do I change the file write behavior or the MDF settings so that I can concatenate or copy files without the CRLF?
EDIT: NEW INFORMATION -- It seems on further review like maybe the minification/whitespace removal is the culprit. If I download the file created by the ADF copy and format it using a JSON formatter, it works. Maybe the CRLF -> LF masked something else. I'm not sure what to do at this point, but its super frustrating.
Other possibly relevant information:
Both the source and sink JSON datasets are set to use UTF-8 (not default(UTF-8), although I tried that). Would a different encoding fix this?
I have tried remapping schemas, creating new data sets, creating new Mapping Data Flows, still get the same error.
EDITED for clarity based on comments:
In the case of a single JSON element in a file, I can get this to work -- data preview returns same success or failure as pipeline when run
In the case of multiple documents merged by ADF I get the below instead. It seems on further review like maybe the minification/whitespace removal is the culprit. If I download the file created by the ADF copy and format it using a JSON formatter, it works. Maybe the CRLF -> LF masked something else. I'm not sure what to do at this point, but its super frustrating.
Repro: Create any valid JSON as a single file, put it in blob storage, use it as a source in a mapping data flow, to do any sink operation. Create a second file with same schema, get them both to run in same flow using wildcard paths. Use a Copy Activity with Merge Files as the Sink Copy Activity and Array of Objects as the File pattern. Try to make your MDF use this new file. If it fails, download the file created by ADF, run it through a formatter (I have used both VS Code -> "Format Document" from standard VS Code JSON extension, and VS 2019 "Unminify" command) and reupload... It should work now.
don't know if you already solved the problem: I came across the exact same problem 3 days ago and after several tries I found a solution:
in the copy data activity under sink settings, use "set of objects" (instead of "array of objects") under File Pattern, so that the merged big JSON has the value of the original small JSON files written per line
in the MDF after setting up the wildcard paths with the *.json pattern, under JSON Settings select: Document per line as the Document form.
After that you should be good to go, as least it solved my problem. The automatic written CRLF in "array of objects" setting in the copy data activity should be a default setting and MSFT should provide the option to omit it in the settings in the future.
According to my test:
1.copy data activity can't change unix(LF) to windows(CRLF).
2.MDF can also parse unix(LF) file and windows(CRLF) file.
Maybe there is something else wrong.
By the way,I see there is a comma after "name":"Customer Name" in your Original file,I delete it before my test.
I have a folder for Octave M-files in C:\\Users\Dropbox\Octave, under which are various subfolders by function categories (normal distribution, chisq...). I just started making those subfolders and they will keep changing (adding, removing, reshuffling) as time goes on.
I would just like to set that folder as root and have Octave search for functions recursively there, just like you set a classpath in Java and JVM searches all folders there.
I used addpath(genpath('C:\\Users\Dropbox\Octave')), but the paths generated are then fixed, not reflecting subsequent subfolder changes.
Shall I add addpath(genpath('C:\\Users\Dropbox\Octave')) to the .octaverc file?
I think there is some confusion here. There are several ways to interact with the path, but for the most part these do not result in permanent changes, unless you save this somehow.
Simply adding a path for an existing octave session will not result in any permanent changes to the usual path that octave initialises at startup. Therefore when you say:
I used addpath(genpath('C:\Users\Dropbox\Octave')), but the paths generated are then fixed, not reflecting subsequent subfolder changes.
this makes no sense, because as soon as you exit your octave session, those added paths should have been gone altogether, and not appear in later octave sessions.
It is more likely that at some point you added these paths, and then used the savepath command, which resulted in your custom paths being added to your .octaverc file.
If that is the case, then yes, you can expect that octave will not "update" what was written in your .octaverc file, unless you call savepath again with an updated path definition.
If you would like the addpath(genpath('C:\Users\Dropbox\Octave')) command you mentioned to be called every time octave starts, so that the current/updated directory structure is loaded, then yes, the best way to do it would be to add that command to your .octaverc file. Make sure you remove the lines in your .octaverc that refer to the previous changes made by savepath. Note that there may be several levels of octaverc files that you need to check (see the relevant page in the manual)
Alternatively, you could simply make sure that this line appears in every script you want to call which intends to make use of those files.
While you may consider this last approach tedious, programmatically it is the most recommended one, since it makes dependencies clear in your code. This is especially important if you ever plan to share your code (and doubly so if you'd like it to be matlab compatible).
PS. All the above mostly applies to matlab too, with the exception that a) matlab's savepath saves path information in a file called pathdef.m, rather than directly in your startup files, and b) matlab uses startup.m instead of .octaverc as startup files. Also, if you don't care about doing this programmatically, matlab provides pathtool, which is a graphical interface for adding / saving directories to the matlab path.
Let's say that i have file /home/foo/myfile without extension. Is there option to add syntax setting into this file? In vim it's :set syntax=javascript. I know that in Sublime you can set default syntax color.
There's similar question Changing default syntax based on filename but there you set specific filename. I need to set it in file itself, since i have a lot of different files without extension.
The package ApplySyntax should be able to do what you want.
ApplySyntax is a plugin for Sublime Text 2 and 3 that allows you to
detect and apply the syntax of files that might not otherwise be
detected properly. For example, files with the .rb extension are
usually Ruby files, but when they are found in a Rails project, they
could be RSpec spec files, Cucumber step files, Ruby on Rails files
(controllers, models, etc), or just plain Ruby files. This is actually
the problem I was trying to solve when I started working on this
plugin.
Set your rules/filenames in the ApplySyntax.sublime-settings file:
// "rules" is a list (array) of checks that you want to make against
the file in the current view. A rule is either a regular expression
or a function. If using a regular expression, you can specify
whether you want it tested against the "file_name" or the first
line of the file (think shebangs and xml files). If the rule is a
function, you must provide the path to the file containing the
function and the name of the function to call. When this function is
called, the "file_name" will be passed to it as the only argument.
You are free to do whatever you want in your function, just return
True or False.
you can do this on the fly through the command menu (on OSX cmd+shift+p, windows ctrl+shift+p) then type what you need (e.g. javascript) and it will come up in the list set syntax: JavaScript.
Sublime will remember this until you close the file
My question is very similar to this one.
I have the files under Mercurial version control and each file have a string:
<modified-time>some time</modified-time>
This line have created the an external tool, and the tool change this line periodically. I don't want to know about changes in this line and want to ignore the changes in this line when I doing 'diff', 'commit' and 'status' commands.
So, if a file has only one change and the change is that line I need to skip this file in the output of 'hg status' command.
If the file has another changes in the other lines I need to get this file in the output of 'hg status' command.
If I commit the file I need to commit all the changes.
(UPDATE sorry to misread the question and mislead OP.)
If all your files are generated by the external tool, track the source files instead.
If not, and if you can modify the generator, make it use the date keyword as with the bundled keyword extension.
If not (OP's comment indicated so), you could use pre-* hooks to ignore the time change (when it's the only change). It'd be a pain, though, since you have to have a hook for every command that would see the file difference.
Or as OP's comment suggested, revert the file when it has only time change, either manually or automatically (on a timer or subscribe to file system notification).