How to correctly parse SSIS variable expression? - ssis

I created an SSIS variable, which I want to turn into an expression, so I write this as the Expression:
="\\livprodad1.liv.local\UserProfiles$\mike.jones\Documents\Files"
The actual link that I am referencing is:
\livprodad1.liv.local\UserProfiles$\mike.jones\Documents\Files
But I get this message: Expression can not be evaluated
Attempt to parse the expression failed. The expression might contain an invalid token, an incomplete token, or an invalid element. It might not be well-formed, or might be missing part of a required element such as a parenthesis.
What am I doing wrong? How should it be written?

You need to double all the slashes in an expression so
"\livprodad1.liv.local\UserProfiles$\mike.jones\Documents\Files"
becomes
"\\livprodad1.liv.local\\UserProfiles$\\mike.jones\\Documents\\Files"
And if you were intending to encode
"\\livprodad1.liv.local\UserProfiles$\mike.jones\Documents\Files"
it would become
"\\\\livprodad1.liv.local\\UserProfiles$\\mike.jones\\Documents\\Files"
The leading equals sign might point to a different issue as the only explicit location that I can think of where one assigns a value is in the Expression Task. Everywhere else, there's a separate field where you list the variable you are modifying
And if the double quote is part of the expression itself, then you also escape it with a backslash \" Excellent example of escaping both on What is the escape character for SSIS Expression Builder?

Related

Expression cannot be evaluated. in SSIS (IDTSVariableDispenser100 vars, Boolean isSensitive)

Attempt to parse the expression "#[User::IncrementVal]==1? #[User::SrcFolderCases]: #[User::SrcFolderAdd]: #[User::SrcFolderDeath]" failed. The expression might contain an invalid token, an incomplete token, or an invalid element. It might not be well-formed, or might be missing part of a required element such as a parenthesis.
Syntax of SSIS conditional operator is the following
boolean_expression?expression1:expression2
In your case, the third variable after :, i.e. #[User::SrcFolderDeath] makes the expression void. Either remove it or transform with concatenation with other variable, for example.

Deal with long numbers in scientific notation in json string - Freemarker

I have a json string which contains a long number but in scientific notation (like 1.559101974041E12 instead of 1559101974041). Due to this, I am not able to parse it using ?eval as this value must be in double quotes in order to get parsed.
I thought of one solution like putting double quotes around them using regex and get them evaluated. After that, use some free marker method to convert value into long. But this solution is very risky and can alter other values as well.
I'm not sure how your template looks, but if you have variable s that contains the string "1.559101974041E12" (the quotation marks aren't part of the string value itself), then you can parse it like s?number. s?eval doesn't work because scientific notation is not part of the FreeMarker syntax (but ?number can parse more formats).
If you will re-print the number in the template, note that depending on locale and configuration settings, it might will look like 1,559,101,974,041. You can prevent that with ?c (for example like ${s?number?c}), in which case it will always look like 1559101974041.

Regex: Is it possible to do a substitution within a capture group?

I have this one line JSON text:
{"schemaText":{"fields":[{"name":"AX_SND_TYPE","type":"string"},{"name":"BWORK","type":"int"}],"name":"XXXSchema","type":"record"},"description":"Autogenerated by NiFi"}
As can be seen there is a property called "schemaText" that contains an object, I want to convert it to a string, so the 'only' thing I need to do is add quotes at the beginning and end of the property and escape the quotes inside.
Using the regular expression bellow (not that my regex knowledge is really low), I am able to do the first step:
({"schemaText":)(\{"fields":\[.*)(,"description.*)
Using the substitution
$1"$2"$3
gives the result:
{"schemaText":"{"fields":[{"name":"AX_SND_TYPE","type":"string"},{"name":"BWORK","type":"int"}],"name":"XXXSchema","type":"record"}","description":"Autogenerated by NiFi"}
But still remains to escape the quotes to get this:
{"schemaText":"{\"fields\":[{\"name\":\"AX_SND_TYPE\",\"type\":\"string\"},{\"name\":\"BWORK\",\"type\":\"int\"}],"name":"XXXSchema","type":"record"}","description":"Autogenerated by NiFi"}
That is have valid JSON format.
The question is: is there a way to escape the quotes inside $2 capture group in the same regular expression?
Thanks in advance.
The answer to your question is no, it's not possible. You're really trying to do two different, unrelated substitutions in a single regular expression. This is a feature that no regular expression engine supports.
Think about it: Your first requirement is for the engine to perform a substitution on the whole text (the quotes), and then, for your second requirement, the engine has to somehow backtrack and perform more substitutions on text which may or may not have already changed: e.g.: It would need to perform a new match on the already substituted text, which, depending on what the first substitution did, may not even exist anymore!
If, as you say, you already have an aproach that works, keep that. A single regular expression is simply not a good fit for what you are trying to do.
I'd recommend tackling this problem using code e.g. with vanilla JavaScript:
let json = '{"schemaText":{"fields":[{"name":"AX_SND_TYPE","type":"string"},{"name":"BWORK","type":"int"}],"name":"XXXSchema","type":"record"},"description":"Autogenerated by NiFi"}';
let obj = JSON.parse(json);
let schemaTextAsString = JSON.stringify(obj.schemaText)
obj.schemaText = schemaTextAsString
var result = JSON.stringify(obj)
You can see this working here.
Note that in your desired output you were not escaping the quotes in schemaText's name field, but this code does.
Finally whenever I use regular expressions I always think of this classic article "Regular Expressions: Now You Have Two Problems"!
Just for your information, you can actually match at every position where a substitution should occur, using an expression such as the following:
/({"schemaText":)|}(,"description")(.*)|([^"]*)"/g
The only issue, as others have mentioned, is that you want to do more than match; you want to perform a "conditional replacement" because there does not exist a single catch-all substitution that will cover all 3 cases you're dealing with (insert starting ", insert \ before quotes, and insert ending ").
You can in fact accomplish this with a single replace() call:
var test = "{\"schemaText\":{\"fields\":[{\"name\":\"AX_SND_TYPE\",\"type\":\"string\"},{\"name\":\"BWORK\",\"type\":\"int\"}],\"name\":\"XXXSchema\",\"type\":\"record\"},\"description\":\"Autogenerated by NiFi\"}";
window.alert(test.replace(/({"schemaText":)|}(,"description")(.*)|([^"]*)"/g, function(a,b,c,d,e){ return (b=="{\"schemaText\":"?b+"\"":(c==",\"description\""?"}\""+c+d:e+"\\\"")) })));
So it's technically "the same regex", but the substitution parameter uses an inline function as replacement rather than a static string.

Regex for matching with and without quotes for dynamic JSON

I have the following text strings:
"Name":"John"}]
"Age":36
"Address":"ABC,PQR234[]/.,#ANYCHARACTERS"
"Gender":null
I need to get two groups (key value pair) from this such that the output would be only:
Key|Value
Name|John
Age|36
Address|ABC,PQR234[]/.,#ANYCHARACTERS
The requirement is to have a single regex to grab everything in the double quotes if the double quotes are present. If not, take the value without the quotes.
In our example above, 36 and null are the one without the quotes and they need to be captured as well.
I have tried a lot but have failed to do so.
UPDATE:
I don't know why I am getting down votes for this question. Yes this is JSON that I am trying to parse but there is a reason behind why I am doing this and not using any document parser.
I am supposed to use Talend for getting a dynamic JSON converted into Key Value Pair. What I mean by dynamic is the fields of the JSON can vary and hence I do not have a fixed schema and hence cannot use a document parser (which demands a fixed structure of JSON). I am devising a solution to get around this using Normalizer (on comma) and then extracting the key value pair which will be in double quotes using Regular Expressions. I tried many things on my own and since I am not an expert in Regular expressions, I have come here to get inputs.
If you know any better solution to this, I would be very happy to get your inputs.
How about this?
/"?([^\n"]*)"?:"?([^\n"]*)"?/
Explained in detail at:
https://regex101.com/r/UM0rl2/1/

How can I populate a query string variable to a text box which contains &,\ and $ in it

I have a variable like say A= drug & medicare $12/$15.
I need to assign it to a text box, but only 'drug' is posted the server. The rest of the data gets truncated.
this.textbox.text= request.querystring["A"].tostring();
The following is not valid for a="foo&bar$12":
http://example.com?a=foo&bar$12
The & symbol is a reserved character, it seperates query string variables. You will need to percent encode a value before sending them to that page.
Also & is a reserved character in HTML/XML. I suggest reading up on percent encoding and html encoding.
I believe you have problems with HTML entities. You need to read up on HTML escaping in your tool of choice. & cannot stand in HTML, since it begins an entity sequence - it needs to be replaced with &. Without specifying at least which toolchain you're using (as per #Richard's comment), we can't really suggest the best way to do it.
EDIT: Now that I reread your question, it seems A is not a variable but a query parameter :) Reading comprehension fail. Anyway, in this case a similar problem exists: & is not a valid character for a query parameter, and it needs URL escaping. Again, how exactly to do it is in the documentation for your toolchain, but in essence & will need to be replaced by %26. Plus sign is also not permitted (or rather it has another meaning); others are tolerated (but there are nicer ways to write them).
That looks more or less like ASP.NET pseudocode, so I'm going to diagnose your problem as the query string needing to be URL encoded. Key/value pairs in the query string are separated by an ampersand (&), and ASP.NET (along with other web platforms) automatically parse out the key value pairs for you.
In this case, the ampersand terminates the value of the "A=..." key/value pair. The problem will be solved if you can URL encode the link that brings the user into your page. If actually using ASP.NET, you can use the HttpUtility.UrlEncode() method for that:
string myValue = Server.UrlEncode("drug & medicare $12/$15");
You'll end up with this querystring instead: A=drug%20%26%20medicare%20%2412%2F%2415