I have a flow. In that flow, there is a case statement based on Document type.
After this case statement there will be an object populated with some metadata. I want to store this object against a variable. So that i can create the object later in the flow, no matter what case was selected.
I have two issues:
To set an object variable, i can't use the Word Template object, because it may be one of two.
My question on this is, how would I populate the variable with the correct object that was just populated in the Statement?
When i try to create a document using the variable with the object, i can't open the document. Can you create an object using an object that is stored in a variable?
After i create a document using the object stored in the variable, this is what I get. I can only assume that the create file method doesn't like using object variables?
Any help would be much appreciated.
Related
I have ADF CopyRestToADLS activity which correctly saves json complex object to Data Lake storage. But I additionally need to pass one of the json values (myextravalue) to a stored procedure. I tried referencing it in the stored procedure parameter as #{activity('CopyRESTtoADLS').output.myextravalue but I am getting error
The actions CopyRestToADLS refernced by 'inputs' in the action ExectuteStored procedure1 are not defined in the template
{
"items": [1000 items],
"count": 1000,
"myextravalue": 15983444
}
I would like to try to dynamically reference this value because the CopyRestToADLS source REST dataset dynamically calls different REST endpoints so the structure of JSON object is different each time. But the myextravalue is always present in each JSON call.
How is it possible to refernce myextravalue and use it as a parameter?
Rich750
You could create another lookup active on REST data source to get the json value. Then pass it to the Stored Procedure active.
Yes, it will create a new REST request, and it seams to be an easy way to achieve your purpose. Lookup active to get the content of the source and won't save it.
The another solution may be get the value from the copy active output file, after the copy active completed.
I'm glad you solved it by this way:
"I created a Data Flow to read from the folder where Copy Activity saves dynamically named output json filenames. After importing schema from sample file, I selected the myextravalue as the only mapping in the Sink Mapping section."
I'm new to Azure Data Factory. I've been able to generate a set of JSON files from a REST API source using a Pipeline. Each file consists of one top level JSON object with an array of up to 100 child objects. The output is saved to an Azure Blob Storage container.
I now want to use a Mapping Data Flow to modify the JSON before I write it to Azure SQL, however I'm struggling with the syntax. I've configured the source to point to the directory containing the JSON files. The Source Projection tab displays the correct schema. I can preview the data and I see a row for each file and I can expand the child objects to see the full structure.
However, when I add a Derived Column action, the Input Schema is blank in the Expression Builder. I can refer to the top level elements in the source using the byName and byPosition functions, but I don't know how I can reference the child elements.
The examples that I have been able to find online use a SQL table or CSV file as a source. I can't find any examples that use hierarchical data as the source for a derived column.
Am I missing something? Is this scenario supported?
I found a way to achieve what I want. This may not be the best approach, but it works.
It seems that it is difficult to deal with JSON that has multiple hierarchies as a source for copy data activities. You can choose one level of repeating data to map to a table structure (the Collection Reference property on the Mapping tab).
In my scenario, there was additional repeating data within the data I was mapping to my table. I updated the mapping to write the child JSON data to a text field in my SQL table. To do this, I needed to use the Azure Data Factory JSON editor for my pipeline. You can access this from the "Code" link in the top right corner of the pipeline visual editor.
I added the following line after the closing bracket for the "mappings" array for my copy activity:
"mapComplexValuesToString": true
The full path to the mapping array in the activity definition is typeProperties - translator - mappings. Make sure your commas are correct after you add the new element.
With this approach, I had a row in my SQL table for each array item in my Collection Reference. The scalar child elements in the array items are mapped to table columns and the child JSON element is written to a data column in the same table.
To extract the values I need within the child JSON, I created a SQL view that uses the CROSS APPLY OPENJSON syntax. This allows me to treat the JSON in the data field similar to a related table. You can specify the structure that your JSON is in. If you have nested data in your JSON, you can apply the same approach for each level.
The OPENJSON command is only supported by more recent versions of SQL Server. I'm using Azure SQL, so that works for me.
My SSIS package fails with the error:
The type of the value being assigned to variable differs from the
current variable type.
I declared a variable of type string and corresponding column name in SQL table is varchar(33). If I use data type as object it succeeds but again i need to use the variable value in an expression for connection string and there object type is not supported.
Please help me how to proceed.
Note: The goal of my SSIS package is to get server list from a table (that's where it fails) and execute some script with a foreach loop container in all the servers.
If I'm reading your question right, what you actually need are two variables: one is an Object variable that receives the data from your SQL table, and one is a String value that you will pass to your ForEach loop.
The difference between them is that your Object variable can hold multiple rows, while your String value can only hold a single item at a time. What you will do is declare an Object variable, populate it with a list of the servers, then use a ForEach loop to step through each item in the list, feeding the values one at a time to your String variable. You will then use the String variable to set up the script you mentioned, and execute it once per server.
For further reading, there is an excellent walkthrough here that will give you screenshots and examples of what I am talking about.
I know i am replying to this thread very late, below explanation is for people who see this in future.
I encountered this situation recently. I did the below
Firstly my Record Set Destination Variable of object type. Using For Each Loop container, i mapped each and every row data(with multiple columns) to an variable(s) of OBJECT type. When i use any other datatypes other than OBJECT datatype for variable(s) i got the error as below
Error: ForEach Variable Mapping number 3 to variable "User::EMPID" cannot be applied.
Then I used the variable(s) of object type in my Execute SQL Task and it inserts data successfully without any error.
Thumb Rule: Map Record Set Destination output to the variable of
OBJECT type.
I encountered this situation recently. I did the below
Firstly my Record Set Destination Variable of object type.
Using For Each Loop container, i mapped each and every row data(with multiple columns) to an variable(s) of OBJECT type. When i use any other datatypes other than OBJECT datatype for variable(s) i got the error as below
Error: ForEach Variable Mapping number 3 to variable "User::EMPID" cannot be applied.
Then I used the variable(s) of object type in my Execute SQL Task and it inserts data successfully without any error.
Thumb Rule: Map Record Set Destination output to the variable of OBJECT type.
In SSIS, i want to read data from a flat file(Flat File Source) and assign certain column values to a variable in SSIS (like IDNo). I need this ID to retrive other information like email and other details to send email dynamically later in the process.
I did some reasearch i found that Script Component is one of the way to assign values to variables, but in our environment script component is not available. Is it possible to do that if so, can some one please provide me a sample. thanks.
You don't necessarily need script component to achieve this.
Read your IDs to a recordset (that would be a variable of type Object):
Create a variable of type Object.
In Data Flow Task create a Recordset Destination.
Pick your variable in VariableName dropdown.
Create Foreach Loop Containter:
Choose Foreach ADO Enumerator in Enumerator setting.
As ADO object source variable pick your recordset variable from the dropdown list
In Variable Mappings assign your email variable with index 0
Send the emails from the loop using email variable as an expression in Send Email Task.
Why is script component not available? It'd make things easier and help to avoid some problems; I don't use mail task anymore in favour of script task, because I ran to some problems with it in the past.
Hope this helps.
If I make changes to an existing linq object by assigning a "new" object of the same type (with different values), SubmitChanges does not make the changes in the database. why not?
existing= new Data.Item{a=1, b=2...};
vs
existing.a= 1;
existing.b= 2;
Because you are not changing the object, you are assigning a new object to the variable.
You need to assign to fields one by one, (or InsertOnSubmit... but that will create a new object in the database and it does not sound like that is what you want to do).
This approach will sort of work if you we assigning the newly created object to a field of an object that LINQ to SQL knows about, but once again, that would be creating a new object rather than changing the one that field previously pointed to (which could result in a bunch of garbage rows in your database if you never get rid of them).