When executing an Input Step in an MDI transformation the log_field entry in the transformation_transformation logging table is show that the injected SQL statement has the '<' sign translated to the http equivalent '%lt;'. However, in the actual execution of the generated transformation after MDI the SQL statement is correct.
Before MDI
After MDI
This appears to be some type of logging issue, but has anyone else encountered this and know if it's a known issue or if there is a workaround or fix for it?
I have executed this through both calling kitchen.sh through the command line and through spoon.sh UI with the same results. I would expect that the '<' signs would not be translated since this is a straight text injection from a JSON file.
Using Pentaho 9.3.0.0-428
Related
I am developing a PDI transformation, which takes data from a MySql database, and output the data into an MSSQL table. But before output, I add a deletion step to delete records in dest. table with same key field values. But I do not know why that by this setting the transformation always fails casting exception of connection timeout of data source.
But, after I added a "Block" step between "table input" and "Delete", the issue got gone, and the transformation got successfully finished.
My configuration and exception message are as blow:
Transformation setting and system exception message
Data Input SQL, and Delete condition
Error what I see from the screen-shot you attached and also recommendation in the 4th error line from top "consider raising value of 'net_write_timeout' on the server"
Default value will be 60, Kindly increase the value for the same.
Follow below document for more reference.
https://wiki.pentaho.com/display/EAI/MySQL
I'm using Simba ODBC to create a connection with Google Big Query and using SSIS (Visual Studio 2019) to read and write information on Big Query. The connection works fine and when I use the ODBC Source with the query option, I'm able to get data from Big Query and used it inside SSIS. But when I use the list of tables, I get an error as below:
Exception of HRESULT: 0xC0014020
Error in Data Flow Task[ODBC Source [100]]: SQLSTATE: 42000, Message: [Simba][BigQuery] (70) Invalid query: Invalid dataset ID ""TEST"". Dataset IDs must be alphanumeric (plus underscores and dashes) and must be at most 1024 characters long.
I believe that this happens because the list of tables appear between ("), instead of (`).
Print of table list
The same happens when I use the ODBC Destination. Is there a way to change the format in which the table list appears ?
Obs.: On the Visual Studio 2015 this table list comes with (`) and I can connect with big query just fine.
I can see that the tool is sending "TEST" as the dataset, however, depending on if Visual Studio is using StandardSQL or LegacySQL, the dataset should be specified as:
# LegacySQL
FROM [myproject:TEST.TABLE_TEST]
# StandardSQL
FROM `myproject:TEST.TABLE_TEST`
I was wondering if Visual Studio accepts a custom query or can be parameterized to remove the quotes. If this doesn't help, could you please share the query that cause the error? I understand that there is a query option (I'm not familiar with Visual Studio) and it is not clear for me the exact moment when the tool returns the error, screenshot without sensitive information would be appreciated.
UPDATE:
You can review the following checkpoints that could help to verify that the Simba driver is correctly set up and it is not the cause of the reported error:
Installation. Check that you are using the last version of the driver. The last version usually contains improvements on the driver.
ODBC Configuration. For example, the Step 13 of the link you will be able to see a drop-down list with the datasets available and select one as the default. If you don't have issues is this step, then the issue could be in the tool that uses the ODBC connection.
Language Dialect. In here you case change between StandardSQL or LegacySQL as needed, for example, you can force your tool to use LegacySQL and use the characters [ and ] that I explained above.
Connection String. If your tool allows to use a string with the connection, you might want to use it and explicitly indicating the default Dataset (among other driver options).
I am pretty new to SSIS (my first real project) and I have built a package that successfully performs some insert operations on a database. I want to deploy it to multiple environments, so I have created a connection manager whose connection string will be passed in from the command line which will in turn be defined in a command file.
The command that I run from the command line is ...
DTExec /F InitialDatabseImport.dtsx /CommandFile InitialCommandFile.config.txt
In the command file (InitialCommandFile.config.txt) I have the following variable assignment ...
/SET \Package.Variables[User::DBConnectionString].Properties[Value];"\"Data Source=(localdb)\ProjectsV12;Initial Catalog=DatabaseName;Integrated Security=SSPI;Connect Timeout=30;Auto Translate=False;"\"
Running this I get the following error ...
Option "Source=(localdb)\ProjectsV12;Initial" is not valid.
It seems that is delimiting the connection string on spaces rather than semicolons.
I have tried a number of configurations with quotes. Here I changed the quotes in front of "Data" to something that seemed to make more sense ...
/SET \Package.Variables[User::DBConnectionString].Properties[Value];\""Data Source=(localdb)\ProjectsV12;Initial Catalog=MarketingPrefsDB;Integrated Security=SSPI;Connect Timeout=30;Auto Translate=False;"\"
which returned the error ...
The argument "\Package.Variables[User::DBConnectionString].Properties[Value];\"Data" has mismatched quotes.
And this one, I wrapped the entire assignment in quotes ...
/SET "\Package.Variables[User::DBConnectionString].Properties[Value];\"Data Source=(localdb)\ProjectsV12;Initial Catalog=MarketingPrefsDB;Integrated Security=SSPI;Connect Timeout=30;Auto Translate=False;\"";
which returned the error ...
Argument ""\Package.Variables[User::DBConnectionString].Properties[Value];\"Data" for option "set" is not valid.
Has anyone else who has seen this problem been able to figure it out?
Thanks,
G
UPDATE - 2014/07/21
When I put the SET command directly in the command line I am getting a slightly different error ...
Command Line ...
C:\Project>DTExec /F QuaeroInitialImport.dtsx /SET \Package.Variables[User::DBConn
ectionString].Properties[Value];"Data Source=servername;Initial Catalog=DatabaseName;Integrated Security=SSPI;Connect Timeout=30;Auto Translate=False;"
Error Message
Argument ""\Package.Variables[User::DBConnectionString].Properties[Value];\"Data Source=servername;Initial Catalog=DatabaseName;Integrated Security=SSPI;Connect Timeout=30;Auto Translate=False;" for option "set" is not valid.
I've met similar issue and just share my work-around in case others reach here thru google.
My case is:
Deploying the package on SQL Server 2014, and I use some package parameter in the package and met this issue. I've met this if I don't use package parameter on another server running SQL Server 2014 also (both server are running the same version of SQL Server 2014).
I fix this by replacing all spaces in the keywords of the connection space with an underscore (_). The connection work for me on my server like:
Data_Source=xxx.xxx.xxx.xxx,xxxx;Initial_Catalog=ABC_DEF;Provider=SQLNCLI11.1;Persist_Security Info=True;Auto_Translate=False;User_ID=userabc;
Please note even I have forgotten to replace some space in some keyword, it still works.
So I finally change it as below:
Data_Source=xxx.xxx.xxx.xxx,xxxx;Initial_Catalog=ABC_DEF;Provider=SQLNCLI11.1;Persist_Security_Info=True;Auto_Translate=False;User_ID=userabc;
So... it doesn't make me comfortable in this case as I am not sure actually if it is working on a correcting settings... you still have to verify it yourself.
So here we have an error I keep getting in my SSIS package but I can't see what is wrong with the statement. I have even tried another sql statement from a project that works and it still raises the error.
The system is VS 2005 running 64 bit debugger, on XP machine. The project has amongst other things a script task then a sql task, the script task outputs the month value to a variable (Dts.Variables("monthName").Value = month), which I then use to create dynamic table name in SQL statement. I haven't got to the excel sheet bit yet as I am trying to get the sql task stage working.
So i have a variable at package level called SQLTableCreate, and in that I have the properties set to:
Evaluate as Expression = true
Expression = "Create Table "+ #[user::monthName]+"(Column1 DATETIME,Column2 NVARCHAR(255),Column3 NVARCHAR(255),Column4 NVARCHAR(255),Column5 NVARCHAR(255),Column6 NVARCHAR(255),Column7 NVARCHAR(255),Column8 NVARCHAR(255),Column9 NVARCHAR(255),Column10 NVARCHAR(255))"
And when I build the package I get:
Nonfatal errors occurred while saving the package:
Error at Package: The variable "user::monthName" was not found in the Variables collection. The variable might not exist in the correct scope.
Error at Package: Attempt to parse the expression ""Create Table "+ #[user::MonthName]+"(Column1 DATETIME,Column2 NVARCHAR(255),Column3 NVARCHAR(255),Column4 NVARCHAR(255),Column5 NVARCHAR(255),Column6 NVARCHAR(255),Column7 NVARCHAR(255),Column8 NVARCHAR(255),Column9 NVARCHAR(255),Column10 NVARCHAR(255))"
" failed and returned error code 0xC00470A6. The expression cannot be parsed. It might contain invalid elements or it might not be well-formed. There may also be an out-of-memory error.Error at Package: The expression for variable "SQLTableCreate" failed evaluation. There was an error in the expression.
There is also a default SQL statement for the variable SQLTableCreate, which uses the current excel connection manager table name. When I put my dynamic statement in the expression section of properties it fills the value and valuetype property of the SQLTableCreate variable with the message:
The expression for variable "SQLTableCreate" failed evaluation. There was an error in the expression.
It's exactly as the error says
The variable "user::monthName" was not found in the Variables collection
Things in SSIS are case sensitive and Variables are one of those things. Make your expression
"Create Table "+ #[User::monthName]+"(Column1 DATETIME,Column2 NVARCHAR(255),Column3 NVARCHAR(255),Column4 NVARCHAR(255),Column5 NVARCHAR(255),Column6 NVARCHAR(255),Column7 NVARCHAR(255),Column8 NVARCHAR(255),Column9 NVARCHAR(255),Column10 NVARCHAR(255))"
Also, I hope this table design is just a sample and not real. Lack of column names and strong data types is technical debt you don't need to incur at this stage.
I'm attempting to create a shared dataset in SSRS based on a DB2 query. The data source uses the DB2OLEDB provider.
When I enter the query directly as text, it executes correctly:
SELECT BTCH_DT FROM FMSCHEMA.TTRANS_HIST_DETL WHERE REC_NO = 13288604
However, when I click the expression button next to the query window, and enter the following:
="SELECT BTCH_DT FROM FMSCHEMA.TTRANS_HIST_DETL WHERE REC_NO = 13288604"
the query fails with
ILLEGAL SYMBOL "=?<ERR_STMT> <WNG_STMT> GET SQL SAVEPOINT HOLD FREE ASSOCIATE". SOME SYMBOLS THAT MIGHT BE LEGAL ARE: SQLSTATE: 42601, SQLCODE: -104
it appears as if SSRS is trying to pass the expression itself to the DB2OLEDB provider instead of evaluating the expression and passing the results of that evaluation.
I've already checked out IBM's site:
http://publib.boulder.ibm.com/infocenter/db2luw/v8/index.jsp?topic=/com.ibm.db2.udb.doc/core/rsql0100.htm but the information provided for SQSLSTATE 42601 wasn't very helpful: "A character, token, or clause is invalid or missing."
Can anyone shed some light on this? Or point me in the right direction? Thanks!