How to skip specific Execution Plan Steps? - deployit

In the TaskBlockService there is a POST call that one or more steps should be skipped. There is not a good example given how the posted XML (List of String) the paths of the steps to skip.
Tried the following content for the POSTed data:
curl -X POST https://xldeploy.company.com/deployit/tasks/v2/5e917094-d054-4cc7-940e-89d851ca225a/skip
File remove-steps.xml content - sample 1:
<list>
<string>0_1_1</string>
</list>
File remove-steps.xml content - sample 2:
<list>
<string>0-1-1</string>
</list>

The first format you list is right, but you have to make sure you're using a step path and not just the path to a block.
Lets say you get the blocks from your deployment plan with this call.
curl -uadmin:password http://localhost:4516/deployit/tasks/v2/28830810-5104-4ab9-9826-22f66dee265d
This will produce the result:
<task id="28830810-5104-4ab9-9826-22f66dee265d" failures="0" state="PENDING" owner="admin">
<description>Initial deployment of Environments/local/TestApp001</description>
<activeBlocks/>
<metadata>
<environment>local</environment>
<taskType>INITIAL</taskType>
<environment_id>Environments/local</environment_id>
<application>TestApp001</application>
<version>1.0</version>
</metadata>
<block id="0" state="PENDING" description="" root="true">
<block id="0_1" state="PENDING" description="Deploy" phase="true">
<block id="0_1_1" state="PENDING" description="Deploy TestApp001 1.0 on environment local"/>
</block>
</block>
<dependencies/>
If you want to see the steps in block 0_1_1 then you can use this rest call to get the steps.
curl -uadmin:password http://local6/deployit/tasks/v2/28830810-5104-4ab9-9826-22f66dee265d/block/0_1_1/step
<block id="0_1_1" state="PENDING" description="Deploy TestApp001 1.0 on environment local" current="0">
<step failures="0" state="PENDING" description="Execute Command"/>
<step failures="0" state="PENDING" description="Copy File001.txt to Infrastructure/localhost"/>
The steps are numbered within the block starting from 1. So if you are want to skip the step - Copy File001.txt to Infrastructure/localhost the step path is 0_1_1_2. Your XML will look like:
<list>
<string>0_1_1_2</string>
</list>

Related

Referencing Project Parameters in BIML

I've been using Catherine W's post on creating project parameters in BIML with some luck. What I'm having a problem with though is setting the expression of a local parameter equal to the project parameter. It's most likely just an XML formatting issue, but I haven't found any examples of it out on the web and have not figured it out on my own yet. So, any suggestions would be most helpful.
Here's the definition of my project parameters that is in my environment BIML file.
<Projects>
<PackageProject Name="ProjParams">
<Parameters>
<Parameter Name="AgentJobName" DataType="String"></Parameter>
<Parameter Name="LoadType" DataType="String">Full</Parameter>
</Parameters>
</PackageProject>
</Projects>
Then under Packages \ Package I have the Variables. I am defining a user variable named LoadType and setting it to the package variable of LoadType in an expression. (There's something in the package that wouldn't use package parameters so I had to create a user variable) I know the reference to #[$Package::LoadType] is incorrect, but that's what I'm trying to figure out. What should it be to get BIML to put in a package parameter?
<Variables>
<Variable EvaluateAsExpression="true" DataType="String" IncludeInDebugDump="Exclude" Name="LoadType">#[$Package::LoadType]</Variable>
Thanks everyone!
It's working for me
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Projects>
<PackageProject Name="so">
<Parameters>
<Parameter DataType="String" Name="ProjectParameter" >Demo0</Parameter>
</Parameters>
<Packages>
<Package PackageName="so_43721322" />
</Packages>
</PackageProject>
</Projects>
<Packages>
<Package Name="so_43721322">
<Parameters>
<Parameter DataType="String" Name="PackageParameter">Demo1</Parameter>
</Parameters>
<Variables>
<Variable Name="PackageParameter" DataType="String" EvaluateAsExpression="true">#[$Package::PackageParameter]</Variable>
<Variable Name="ProjectParameter" DataType="String" EvaluateAsExpression="true">#[$Project::ProjectParameter]</Variable>
</Variables>
</Package>
</Packages>
</Biml>
I create a project and a package level parameter and then create two variables within my package, each referencing the parameter (#[$Project::ProjectParameter] and #[$Package::PackageParameter])
Am I missing some nuance?

How do you ignore the 1st two lines of a CSV file in Dataweave?

I am using Anypoint Studio 6.1 with Mule 3.8.1 and in the Transform Message component I have an input CSV file which has fieldnames on the 1st line and datatypes for that filed on the 2nd line. I want to start processing the records from line 3 onwards, but cannot configure the Reader Configuration to miss the 2nd line containing the datatypes.
Is there a way I can do this so the 1st line is used for fieldnames, the 2nd line is ignored and the 3rd line onwards are the records that are processed?
Test CSV:
Product,Price
string,decimal(10,2)
phone,99
tv,499
XML flow:
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:dw="http://www.mulesoft.org/schema/mule/ee/dw" xmlns:metadata="http://www.mulesoft.org/schema/mule/metadata" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/ee/dw http://www.mulesoft.org/schema/mule/ee/dw/current/dw.xsd">
<flow name="csvreaderFlow">
<dw:transform-message metadata:id="08c0a5d4-9d7d-44ad-b72b-bac83ab6b6c1" doc:name="Transform Message">
<dw:input-payload doc:sample="sample_data\list_csv.csv" mimeType="application/csv">
<dw:reader-property name="bodyStartLineNumber" value="2"/>
</dw:input-payload>
<dw:set-payload><![CDATA[%dw 1.0
%output application/json
---
payload map ((payload01 , indexOfPayload01) -> {
products: payload01.Product
})]]></dw:set-payload>
</dw:transform-message>
</flow>
</mule>
Even if I increase the body line start number to 10, it still shows the 2 line and the 2 records to process. I thought this would cause an error.
Thanks
In the Input reader configuration for CSV on the Dataweave, they have a setting "bodyStartLineNumber" (set that to '3' in your case). Let me know if you have issues,
Here are the steps to be followed on the Dataweave,
1. Right click on the "Payload" label (left panel).
2. Click on the "Reader Configuration" option.
3. GUI opens with multiple options to be set.
4. Configure the "Body Start Line Number"

How to manage giant fixed-width file in SSIS?

I have a fixed width file that is about 1200 characters wide and has about 300+ columns. I'm looking for a way to create a fixed-width data source in SSIS without using the UI for the flat file connection manager. Is there a way to modify the column definitions without having to use the UI in SSIS? I can't find a file for the data connection anywhere in the project.
Am I doomed to manually add 300+ columns into the flat-file connection manager one by one?
Two options come to mind. The first is to Install BIDSHelper and use the Create Fixed Width Columns
The other, as #ElectricLlama mentioned is to use BIML. This too will require the installation of BIDS Helper but to convert a .biml file into a .dtsx Short Walkthrough
This should approximate creating a package with a flat file connection manager (with a single column) adding a data flow and inside that consume our flat file and wire it up to a Row count. This is approximate for what you want. Just fill in the XML in the Columns tag.
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Connections>
<FlatFileConnection
Name="FF dchess"
FileFormat="FFF dchess"
FilePath="C:\ssisdata\SO\Input\dchess.txt"
/>
</Connections>
<FileFormats>
<FlatFileFormat
Name="FFF dchess"
CodePage="1252"
RowDelimiter="CRLF"
IsUnicode="false"
FlatFileType="RaggedRight"
>
<Columns>
<Column Name="MyColumn" Length="08" DataType="AnsiString" ColumnType="FixedWidth" CodePage="1252" />
</Columns>
</FlatFileFormat>
</FileFormats>
<Packages>
<Package Name="dchess" ConstraintMode="Linear" ProtectionLevel="DontSaveSensitive">
<Connections >
<Connection ConnectionName="FF dchess" />
</Connections>
<Variables>
<Variable Name="CurrentFileName" DataType="String">C:\ssisdata\so\Input\dchess.txt</Variable>
<Variable Name="RowCountInsert" DataType="Int32">0</Variable>
</Variables>
<Tasks>
<Dataflow Name="DFT Load file" >
<Transformations>
<FlatFileSource
Name="FF_SRC dchess"
ConnectionName="FF dchess"
RetainNulls="true">
</FlatFileSource>
<RowCount Name="CNT Source" VariableName="User.RowCountInsert"></RowCount>
</Transformations>
</Dataflow>
</Tasks>
</Package>
</Packages>
</Biml>
Generated package looks like
Feel free to pick your jaw up off the ground ;)

How to execute (via ant) a set of sql files with mysql command line

I want to execute all sql files which resistes in a given directory.
The call i want to make is something like this:
mysql --host=dbbackend --user=stack --password=overflow dbname -e 'source file1.sql'
This can be expressed as:
<apply executable="mysql" dir="." failonerror="true">
<arg value="--host=dbbackend"></arg>
<arg value="--user=stack"></arg>
<arg value="--password=overflow"></arg>
<arg value="mydbname"></arg>
<arg value="--e source dummy.sql"></arg>
<fileset dir="${db.dump.location.data}" casesensitive="no" description="take all sql files">
<patternset>
<include name="**/*.sql" />
</patternset>
</fileset>
</apply>
If i have 3 sql files in that dir, then dummy.sql is called 3 times. So far so good. Is there a placeholder available to change this line:
<arg value="--e source dummy.sql"></arg>
into:
<arg value="--e source ${unknown.placeholder.name}"></arg>
If there is a placeholder, then i want to use it for the "input" attribute of the "apply" tag (and remove the argument "-e").
There is a "srcfile" tag available for the "apply" element, but i can not call this (does not work):
<arg value="--e source"></arg>
<srcfile/>
Do you have suggestions on how to do it with native ant declaration?
Is it possilbe to build a workaround using antcall+fileset (+placeholder)?
The noobish workaround is to iterate via fileset and create a temp sql file with references to the sql files. As a last step: call it static via "-e". But that is a workaround that i want to remove (by this question).
PS: I do not want to use ant-contrib features.
Why not using the ant sql task, which is designed to do exactly what you want to do?
Else, you may use the srcfile nested element:
<apply ... parallel="false">
<...>
<arg value="--e source" />
<srcfile/>
<fileset ...></fileset>
</apply>
There is a complete example on the manual page of the apply task.

Erro loading schemes from Eclipse - Mule IDE

i am attempting to create my first mule server but i get an error for any external scheme i try and include,
my Config file is as follows (working on Eclipse Indigo with mule standalone 3.2 installation):
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns="http://www.mulesoft.org/schema/mule/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:spring="http://www.springframework.org/schema/beans"
xmlns:http="http://www.mulesoft.org/schema/mule/http"
xmlns:vm="http://www.mulesoft.org/schema/mule/vm"
xmlns:quartz="http://www.mulesoft.org/schema/mule/quartz"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/3.2/mule.xsd
http://www.mulesoft.org/schema/mule/quartz http://www.mulesoft.org/schema/mule/quartz/current/mule-quartz.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/3.2/mule-http.xsd
http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/3.2/mule-vm.xsd">
<flow name="ChatListener">
<quartz:inbound-endpoint jobName="eventTimer" repeatInterval="2000">
<quartz:event-generator-job>
<quartz:payload>Poll Chat DB</quartz:payload>
</quartz:event-generator-job>
</quartz:inbound-endpoint>
<component>
<singleton-object class="com.TimeLineListener.ChatListener" />
</component>
<vm:outbound-endpoint path="ChatMsgs" exchange-pattern="one-way"/>
</flow>
<flow name="TimeLineMsgSender">
<composite-source>
<!-- Incoming Chat Msgs -->
<vm:inbound-endpoint path="ChatMsgs" exchange-pattern="one-way"/>
<!-- Incoming SIEM Msgs -->
<vm:inbound-endpoint path="SIEMMsgs" exchange-pattern="one-way"/>
<!-- Incoming NMS Msgs -->
<vm:inbound-endpoint path="NMSMsgs" exchange-pattern="one-way"/>
</composite-source>
<!-- Tested OutPut endpoint -->
<stdio:outbound-endpoint system="OUT"/>
</flow>
</mule>
and the errors i recieve are:
1.
The prefix "stdio" for element "stdio:outbound-endpoint" is not bound. mule-config.xml ‪/ChatTester‬ line 41 XML Problem
2.
cvc-complex-type.2.4.a: Invalid content was found starting with element 'vm:inbound-endpoint'. One of '{"http://www.mulesoft.org/schema/mule/core":abstract-inbound-endpoint}' is expected. mule-config.xml ‪/ChatTester‬ line 31 XML Problem
3.
cvc-complex-type.2.4.a: Invalid content was found starting with element 'quartz:inbound-endpoint'. One of '{"http://www.mulesoft.org/schema/mule/core":description, "http://www.mulesoft.org/schema/mule/core":composite-source, "http://www.mulesoft.org/schema/mule/core":abstract-inbound-endpoint, "http://www.mulesoft.org/schema/mule/core":abstract-message-processor, "http://www.mulesoft.org/schema/mule/core":abstract-outbound-endpoint, "http://www.mulesoft.org/schema/mule/core":response}' is expected. mule-config.xml ‪/ChatTester‬ line 17 XML Problem
any idea what i"m doing wrong?
1.
The prefix "stdio" for element "stdio:outbound-endpoint" is not bound. mule-config.xml ‪/ChatTester‬ line 41 XML Problem
This one is easy: you're missing the stdio namespace declaration.
2.
cvc-complex-type.2.4.a: Invalid content was found starting with element 'vm:inbound-endpoint'. One of '{"http://www.mulesoft.org/schema/mule/core":abstract-inbound-endpoint}' is expected. mule-config.xml ‪/ChatTester‬ line 31 XML Problem
3.
cvc-complex-type.2.4.a: Invalid content was found starting with element 'quartz:inbound-endpoint'. One of '{"http://www.mulesoft.org/schema/mule/core":description, "http://www.mulesoft.org/schema/mule/core":composite-source, "http://www.mulesoft.org/schema/mule/core":abstract-inbound-endpoint, "http://www.mulesoft.org/schema/mule/core":abstract-message-processor, "http://www.mulesoft.org/schema/mule/core":abstract-outbound-endpoint, "http://www.mulesoft.org/schema/mule/core":response}' is expected. mule-config.xml ‪/ChatTester‬ line 17 XML Problem
For these ones: I don't know. Maybe due to the mix of "current" and "3.2" you're using in the namespace locations? Try only with "3.2" instead of current to see if it helps.
Otherwise, nothing visibly crazy in your config :)
Actually, this is a problem of Eclipse and doesn't relate to your configuration. Hope this helps:
"Since Mule’s schemas are not split over multiple schema files, it’s safe to turn off this feature. In Eclipse’s preferences, go to XML > XML Files > Validation and clear the Honour all schema locations check box. Mule config files should now validate without errors again."-MuleSolf Blog.
For more details: http://blogs.mulesoft.org/overcoming-xml-validation-errors-in-eclipse-35/