Question:
Can I use jinja2 string syntax into task_id string or we are limited to f-string?
Example:
Let's say I use a BashOperator and inside I have assigned params. I can use successfully these params into my bash_command, however once I use them into the task_id string, I get the following error:
airflow.exceptions.AirflowException: The key (my_task_id_string{{ params.paramname1}}{{ params.paramname2}}) has to be made of alphanumeric characters, dashes, dots and underscores
Instead, in order to mitigate this, I use an f-string instead like so: f"my_task_id_string{paramname1}{paramname2}"
Not all the parameters in the BashOperator is templated, the same applies for all the operators available in Airflow.
For the BashOperator the fields that Jinja will template on are bash_command and env.
For further reference, you can always refer to any operators source doc-strings, which will indicate if the field is templatable.
For BashOperator, you can refer here https://airflow.apache.org/docs/stable/_modules/airflow/operators/bash_operator.html
Related
Consider the following query:
select '"{\"foo\":\"bar\"}"'::json;
This will return a single record of a single element containing a JSON string. See:
test=# select json_typeof('"{\"foo\":\"bar\"}"'::json); json_typeof
-------------
string
(1 row)
It is possible to extract the contents of the string as follows:
=# select ('"{\"foo\":\"bar\"}"'::json) #>>'{}';
json
---------------
{"foo":"bar"}
(1 row)
From this point onward, the result can be cast as a JSON object:
test=# select json_typeof((('"{\"foo\":\"bar\"}"'::json) #>>'{}')::json);
json_typeof
-------------
object
(1 row)
This way seems magical.
I define no path within the extraction operator, yet what is returned is not what I passed. This seems like passing no index to an array accessor, and getting an element back.
I worry that I will confuse the next maintainer to look at this logic.
Is there a less magical way to do this?
But you did define a path. Defining "root" as path is just another path. And that's just what the #>> operator is for:
Extracts JSON sub-object at the specified path as text.
Rendering as text effectively applies the escape characters in the string. When casting back to json the special meaning of double-quotes (not escaped any more) kicks in. Nothing magic there. No better way to do it.
If you expect it to be confusing to the afterlife, add comments explaining what you are doing there.
Maybe, in the spirit of clarity, you might use the equivalent function json_extract_path_text() instead. The manual:
Extracts JSON sub-object at the specified path as text. (This is functionally equivalent to the #>> operator.)
Now, the function has a VARIADIC parameter, and you typically enter path elements one-by-one, like the example in the manual demonstrates:
json_extract_path_text('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}}',
'f4', 'f6') → foo
You cannot enter the "root" path this way. But (what the manual does not add at this point) you can alternatively provide an actual array after adding the keyword VARIADIC. See:
Pass multiple values in single parameter
So this does the trick:
SELECT json_extract_path_text('"{\"foo\":\"bar\"}"'::json, VARIADIC '{}')::json;
And since we are all about being explicit, use the verbose SQL standard cast syntax:
SELECT cast(json_extract_path_text('"{\"foo\":\"bar\"}"'::json, VARIADIC '{}') AS json)
Any clearer, yet? (I would personally prefer your shorter original, but I may be biased, being a "native speaker" of Postgres..)
The question is, why do you have that odd JSON literal including escapes as JSON string in the first place?
I have a json string which contains a long number but in scientific notation (like 1.559101974041E12 instead of 1559101974041). Due to this, I am not able to parse it using ?eval as this value must be in double quotes in order to get parsed.
I thought of one solution like putting double quotes around them using regex and get them evaluated. After that, use some free marker method to convert value into long. But this solution is very risky and can alter other values as well.
I'm not sure how your template looks, but if you have variable s that contains the string "1.559101974041E12" (the quotation marks aren't part of the string value itself), then you can parse it like s?number. s?eval doesn't work because scientific notation is not part of the FreeMarker syntax (but ?number can parse more formats).
If you will re-print the number in the template, note that depending on locale and configuration settings, it might will look like 1,559,101,974,041. You can prevent that with ?c (for example like ${s?number?c}), in which case it will always look like 1559101974041.
I am trying to write a CLI client in Elixir for an API so that I can login to the API system, fetch the data I need for my calculation and then logout. I have defined a Packet.Login struct that supposed to be my internal data structure that I end up with after parsing the JSON I receive.
I am using Poison to parse the JSON. The problem is that it seems like, because of the API returning capitalised properties, I can't match them when printing or parsing, as Poison will return a map with these capitalized keys. The problem is that it seems impossible for me to use the alias like this. If I try to use another syntax,
packet[:Token]
it still does not work and instead gives me an error. But this time about Packet.Login not implementing the Access behaviour. I can understand that part, but not the first issue. And I'm trying to keep the code stupid simple.
defmodule Packet.Login do
defstruct [:Data, :Token]
end
defimpl String.Chars, for: Packet.Login do
def to_string(packet) do
"Packet:\n---Token:\t\t#{packet.Token}\n---Data:\t#{packet.Data}"
end
end
loginPacket = Poison.decode!(json, as: %Packet.Login{})
IO.puts "#{loginPacket}"
When trying to compile the above I get this:
** (CompileError) lib/packet.ex:31: invalid alias: "packet.Token". If you wanted to define an alias, an alias must expand to an atom at compile time but it did not, you may use Module.concat/2 to build it at runtime. If instead you wanted to invoke a function or access a field, wrap the function or field name in double quotes
(elixir) expanding macro: Kernel.to_string/1
Is there a way for me to fix this somehow? I have thought of parsing the map and de-capitalizing all fields first, but I would rather not.
Why can't I have capitalized keys for a struct? It seems like I can though, as long as I don't try to use them.
In order to access a field of a map which is an atom starting with an uppercase letter, you need to either put the key in quotes, e.g. foo."Bar" or use the bracket syntax, e.g. foo[:Bar]. foo.Bar in Elixir is parsed as an alias. With structs, you cannot use the bracket syntax, so the easiest way is to use quotes around the field name. In your code, you'll therefore need to change:
"Packet:\n---Token:\t\t#{packet.Token}\n---Data:\t#{packet.Data}"
to:
"Packet:\n---Token:\t\t#{packet."Token"}\n---Data:\t#{packet."Data"}"
I could not find this documented clearly anywhere but Elixir's source mentions this in some places and also uses this syntax to access some functions in :erlang which have names that are not valid identifiers in Elixir, e.g. :erlang."=<".
Fun fact: you can define functions in Elixir that can only be called with this quote syntax as well:
iex(1)> defmodule Foo do
...(1)> def unquote(:"!##")(), do: :ok
...(1)> end
iex(2)> Foo."!##"()
:ok
I am trying to use a *.resw file in my UWP app to store localized strings. I am loading these strings through ResourceLoader.GetString() and am placing them in a MessageDialog for presentation to the user, but no matter what I do the return value of GetString() is an empty (zero-length) string. I am following the SDK sample for localization, but am not getting the expected response.
The string I am trying to use is of the format InvalidAssemblyDialog.Message.
As it turns out this problem was due to my using dots in the keys for my strings in the *.resw file. Dots are reserved, and my usage of them was causing name-resolution errors. In the case of the example above, I changed it to InvalidAssemblyDialog_Message.
Here the documentation says "." characters should be replaced with "/" when resources are queried from code.
If a resource name is segmented (it contains "." characters), then replace dots with forward slash ("/") characters in the resource name. Property identifiers, for example, contain dots; so you'd need to do this substition in order to load one of those from code.
I’m trying to use the REST API provided by DeployIt (v3.9) to list all the packages available on a given project.
Thus, I use the GET /repository/query service
So, I’m calling this service with the following URL:
http://[server]/deployit/repository/query?namePattern=my-app&type=udm.DeploymentPackage
Unfortunately, I don’t get anything (just an empty list).
If I remove the namePattern from my URL, then I get a long list of all applications (not only the only I'm interested in).
So it appears that I don’t set correctly the namePattern attribute. In the documentation, they say:
a search pattern for the name. This is like the SQL "LIKE" pattern:
the character '%' represents any string of zero or more characters,
and the character '_' (underscore) represents any single character.
Any literal use of these two characters must be escaped with a
backslash ('\'). Consequently, any literal instance of a backslash
must also be escaped, resulting in a double backslash ('\').
So I tried the following URL:
http://[server]/deployit/repository/query?type=udm.DeploymentPackage&namePattern=my-app : empty list
http://[server]/deployit/repository/query?type=udm.DeploymentPackage&namePattern=%my-app%: error 400
http://[server]/deployit/repository/query?type=udm.DeploymentPackage&namePattern=%25my-app%25 (trying to escape the % character): empty list
http://[server]/deployit/repository/query?type=udm.DeploymentPackage&namePattern=Applications/my-app/2.0.0 (with a real version): error, character ‘/’ not allowed.
http://[server]/deployit/repository/query?type=udm.DeploymentPackage&namePattern=2.0.0 : I get the list of all applications deployed with a version 2.0.0 (including my my-app), but that's not what I'm looking for (I want all versions available on DeployIt for my-app).
So, what is the correct URL to retrieve the list of deployed applications?
I've solved my problem. In fact, the namePattern only applies to the last part of the Application name, i.e. the version.
Thus, I have to use the parent attribute to retrieve the list of my application:
http://[server]/deployit/repository/query?type=udm.DeploymentPackage&parent=Applications%2Fmy-app&resultsPerPage=-1