Airflow Jinja template for variable with default value - jinja2

According to the official docs if you want to use Jinja to fetch a variable and provide a default value, the syntax is:
var.value.get('my.var', 'fallback')
But in running that on version 1.10.6, I get the following error:
ERROR - 'Variable get does not exist'
That is, the parser thinks that I'm looking for a variable named get instead of treating get() as a function call.
What is the correct syntax for using Jinja to fetch a variable with a default value in case the variable is not set in the server?

The usage of template {{ var.value.get('my.var', 'fallback') }} is working for me in Airflow 2.1.

Related

TCL - Reading console output (not keystrokes)

So I am very new to TCL commands and I have to write a simple query which would read the hash value generated by a command.
TCL query is part of a bigger script where we need to generate a hash value using tcl command.
Below is the whole scenario:
Basically, need to execute a command to generate HASH value.
For ex:
request password-hash -password <password_value>
Once above command is executed, the shell will provide a hash value. This hash value then should be provided to another command.
For ex:
set password-hash <above generated value here>
After a lot of searching I think exec command will give the hash as output, I was then planning to store it in some variable using set command, something like below:
set hash_value [exec request password-hash -password <password_value>]
& then
set password-hash $hash_value
however, I am facing error that tcl evaluation failing.
The script 'set hash_value [exec request password-hash -password <password_value>];' evaluation failed. Error: System.ComponentModel.Win32Exception (0x80004005): The system cannot find the file specified
Is exec command correct way of doing the what I actually wanted to do?
Since evaluation is failing, I think some syntax issue is there maybe?
Any better way to read console output and provide it as input to another command?
Thanks!!

mysql-connector-python adds single quotes around table name when table is passed as argument. Table name comes from Flask session variable

I am new to MySQL and I am building a Flask project and using mysql.connector to query a MySQL Database. I know this question has been answered many times before but this is more specific to using MySQL with Flask.
I need to pass a query where I want to plug in the table name into the query, dynamically, depending on the value stored in the session variable in Flask. But the problem is, if I try to do:
Method 1:
cur.execute('SELECT * FROM %s;',(session['table_name'],))
the database throws an error stating that such a table is not found. However, the problem is mysql.connector keeps enclosing the table name with single quotes, hence the error.
Sample Error Statement:
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''52_data'' at line 1
Here the table name should be 52_data and not '52_data'.
Only other workaround, I figured, is using:
Method 2:
cur.execute('SELECT * FROM '+session['table_name']+';')
which is working but it does not escape SQL Injection, I am guessing, since it's direct concatenation, unlike Method 1, where the cur.execute() function handles the escaping, as per this question.
The value being passed is stored in a sessions variable in Flask, which is not so secure, as per Miguel's Video. Hence, I want to escape that string, without triggering off an error.
Is it possible to implement Method 1 in a way that it does not add the quotes, or maybe escape the string using some function? Or maybe any other Python/Flask package that can handle this problem better?
Or if nothing works, is checking for SQL Injection manually using regex is a wiser option?
Thanks in advance.
Note: The package name for this mysql.connector is mysql-connector-python and not any other same sounding package.
For identifiers, you can use something like:
table_name = conn.converter.escape(session['table_name'])
cur.execute('SELECT * FROM `{}`'.format(table_name))
For values placeholders, you can use your Method 1, by using the parameters in the cur.execute() method. They will be escaped and quoted.
More details in https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-execute.html
NOTE: You don't need to end the SQL statements with ;

SSIS ExecuteSQL task error with ResultSet

Hi I am trying to return a ResultSet from an "Execute SQL Task" in SSIS.
I am then trying to save the result in a "ResultSet" variable.
My Query works in the console and my connection is ok to.
Here is my Query
SELECT Src FROM [myDB].[dbo].[myTable] group by Src
Src is nvarchar(255)
When I execute the task I am getting following error
[Execute SQL Task] Error: Executing the query "SELECT Src
FROM [myDB].[dbo].[myTable..." failed with the following error: "The type of the value (DBNull) being assigned to variable "User::ResultSet" differs from the current variable type (String). Variables may not change type during execution. Variable types are strict, except for variables of type Object.
". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
I am also attaching the screen shots for the Task setup.
Please help.
ExecuteSQL Screen1
ExecuteSQL Screen2 - ResultSet setup
In SSIS 2012 within a ForEach loop using an ADO enumerator, you are unable to assign a null value to a variable of type String when the source value came from a SQL query returning a null value.
I would suggest Derived Column Transform feature of SSIS
you'll use the SSIS ISNULL function to detect the NULL value - and then do something with it.
More info about Derived Column Transform
For example transforming String column:
ISNULL([String-Column]) ? "There Was No Value" : [String-Column]

Why does variable of Data Type SByte using expression evaluate to 0?

I'm trying to set some variables of type SBYTE to null at runtime using an expression directly on the variable. The variable is scoped to the control flow.
Why does the variable evaluate to 0 instead of null? How can I set it to null?
This is because SSIS variables cannot be NULL. Oddly, I couldn't find anything in the documentation that explicitly says that they can't be NULL, it just says
The value of a user-defined variable can be a literal or an expression
But an (old) Microsoft blog entry says "If SSIS variables supported NULL values..." and the Professional Microsoft SQL Server 2012 Integration Services book says:
In SSIS, variables can't be set to NULL. Instead, each variable data
type maintains a default value in the absence of a value.
And indeed, if you evaluate just NULL(DT_UI1) you get zero; evaluating NULL(DT_BOOL) gives False; evaluating NULL(DT_DATE) gives 12/30/1899 12:00:00 AM.
And using an expression to set a variable to NULL doesn't work either: if you create an Int32 variable, set EvaluateAsExpression to True and enter the value NULL(DT_UI4) then you get the error "Property value is not valid", with the detailed message "NULL(DT_UI4) is not a valid value for Int32".

SSIS can't use variables (Flat File > OLE DB Command), Must declare the scalar variable "#"

Apologies, I'm an SSIS noob and I'm struggling with trying to import a flat file and do an OLE DB command WHILE including a variable in the SqlCommand.
You can see my query and the attempt to include an already defined variable. The error is displayed below:
Must declare the scalar variable "#" I understand what the error is saying, but after hours of searching, I cannot figure out how to use variable in the OLE DB Command.
Thanks in advance for any assistance.
Try putting the ssis variable [user::ClientID] into a derived column and add it to the data flow. You should be able to use a ? and assign that value to your query just like any other value in your data flow.