BCP SQL exporting to CSV - mysql

Im trying to run a Store procedure query to export to CSV file using bcp as a daily task on SQL server
using a normal query works fine for example
select #csv = 'bcp "select * from Table" queryout '+#FileAndPath2+' -c -t, -T -S' +##servername
However when I add my query which is a list of transactions data within a date range it seems to crash
#p_companyId uniqueidentifier = '189be460-99d1-42e9-b4ed-8de6f8724ce8',
#p_Path varchar(300) = 'C:\temp\','
#p_Date datetime = getutcdate
set #FileAndPath2=#p_Path + CONVERT(nvarchar(30), #p_Date, 112) + '_' + CONVERT(varchar(36), #p_companyId) + '_transactionslog.csv';
declare #csv varchar(8000)
declare #csvSQL varchar(8000)
set #csvSQL = 'SELECT TOP (100) [KICSDEV].dbo.MOVIEDETAIL.Title , [KICSDEV].dbo.MEMBERMOVIEPURCHASELOG.MemberId, [KICSDEV].dbo.MEMBERMOVIEPURCHASELOG.CreateDateTime as ''DateTime'' FROM [KICSDEV].dbo.MEMBERMOVIEPURCHASELOG INNER JOIN [KICSDEV].dbo.MOVIEDETAIL ON [KICSDEV].dbo.MEMBERMOVIEPURCHASELOG.MovieDetailId = [KICSDEV].dbo.MOVIEDETAIL.MovieDetailId INNER JOIN [KICSDEV].dbo.MEMBER ON [KICSDEV].dbo.MEMBERMOVIEPURCHASELOG.MemberId = [KICSDEV].dbo.MEMBER.MemberId INNER JOIN [KICSDEV].dbo.CINEMA ON [KICSDEV].dbo.MEMBER.CinemaId = [KICSDEV].dbo.CINEMA.CinemaId WHERE ([KICSDEV].dbo.CINEMA.CompanyId = '+ #p_companyId + ' and [KICSDEV].dbo.MEMBERMOVIEPURCHASELOG.CreateDateTime >= ' + #p_Date +' and [KICSDEV].dbo.MEMBERMOVIEPURCHASELOG.CreateDateTime < DATEADD (day , 1 , '+#p_Date+'))'
select #csvSQL
select #csv = 'bcp "'+ #csvSQL +'" queryout '+#FileAndPath2+' -c -t, -T -S' +##servername
exec master..xp_cmdshell #csv
When I run it comes up as "The data types varchar and uniqueidentifier are incompatible in the add operator." error
When i change the Company to the string instead of the variable in the query it works fine but errors on this.
set #csvSQL = 'SELECT TOP (100) [KICSDEV].dbo.MOVIEDETAIL.Title , [KICSDEV].dbo.MEMBERMOVIEPURCHASELOG.MemberId, [KICSDEV].dbo.MEMBERMOVIEPURCHASELOG.CreateDateTime as ''DateTime'' FROM [KICSDEV].dbo.MEMBERMOVIEPURCHASELOG INNER JOIN [KICSDEV].dbo.MOVIEDETAIL ON [KICSDEV].dbo.MEMBERMOVIEPURCHASELOG.MovieDetailId = [KICSDEV].dbo.MOVIEDETAIL.MovieDetailId INNER JOIN [KICSDEV].dbo.MEMBER ON [KICSDEV].dbo.MEMBERMOVIEPURCHASELOG.MemberId = [KICSDEV].dbo.MEMBER.MemberId INNER JOIN [KICSDEV].dbo.CINEMA ON [KICSDEV].dbo.MEMBER.CinemaId = [KICSDEV].dbo.CINEMA.CinemaId WHERE ([KICSDEV].dbo.CINEMA.CompanyId = ''189be460-99d1-42e9-b4ed-8de6f8724ce8'' and [KICSDEV].dbo.MEMBERMOVIEPURCHASELOG.CreateDateTime >= ' + #p_Date +' and [KICSDEV].dbo.MEMBERMOVIEPURCHASELOG.CreateDateTime < DATEADD (day , 1 , '+#p_Date+'))'
Conversion failed when converting date and/or time from character string.
I think it something to do with all the delimiters and data types.

Several problems here. Think of the problem this way. You are building a command line parameter for a command line utility, so everything has to be built into a string.
Step 1: Make sure everything is a string before concatenating the query
You are missing some casts Cast(#p_companyId as VarChar(36)) and CAST( #p_Date as VarChar(25)) you also need to quote the cast of the company id and dates in the formatted string. I recommend making a new variable to have the UTC date as a string #p_DateAsStr varchar(25) = CAST( #p_Date as VarChar(25) ), instead of repeating that over and over.
Step 2: String values in the query need to be quoted
Since you are calling BCP you have to format the query as a string, string parameters need to be quoted. You have to use the single quotes because the string to BCP is in double quotes, for instance:
set #csvSQL = 'SELECT .... WHERE CompanyId = '''+ Cast(#p_companyId as VarChar(36)) + ''....'
Step 3: Convert any strings in the query back to native types as needed by built in functions
We are OK with the GUID specified as a string (if we quote it), but for DataAdd we need to convert back from string to date Like this
CreateDateTime < DATEADD (day , 1 , CAST(''' +#p_DateAsStr+''' as DateTime))
[Update] added a quoted string for the date

Related

How to Convert DATETIME to CHAR in MS SQL

I need to convert a MS SQL date time a specific format:
MM/DD/YYYY HH:MM AMPM
which means that the HH has to have a leading zero if necessary: 03:25 PM instead of 3:25 PM.
Also, there should be a space between the minutes and either AM or PM.
I couldn't find one of the convert codes to match this.
In case it matters, this is SQL Server 2008 R2.
Use the new FORMAT function:
DECLARE #dt DATETIME = '2016-04-18 15:05:22'
SELECT FORMAT(#dt, 'MM/dd/yyyy hh:mm tt')
-- output: 04/18/2016 03:05 PM
Available from SQL Server 2012.
Reference: https://msdn.microsoft.com/en-us/library/ee634398.aspx
Examples: http://sqlhints.com/2013/06/23/format-string-function-in-sql-server-2012/
SELECT CONVERT(NVARCHAR,GETDATE(),101) + ' ' +
CASE SUBSTRING(CONVERT(NVARCHAR,GETDATE(),100),13,1) WHEN ' ' THEN '0' ELSE SUBSTRING(CONVERT(NVARCHAR,GETDATE(),100),13,1) END +
SUBSTRING(CONVERT(NVARCHAR,GETDATE(),100),14,4) + ' ' + RIGHT(CONVERT(NVARCHAR,GETDATE(),100),2)
Might I also suggest this code:
DECLARE #OFDate DATETIME
SET #OFDate = DATEADD(hh,13,GETDATE())
SELECT CONVERT(NVARCHAR,#OFDate,101) + ' ' +
CASE SUBSTRING(CONVERT(NVARCHAR,#OFDate,100),13,1) WHEN ' ' THEN '0' ELSE SUBSTRING(CONVERT(NVARCHAR,#OFDate,100),13,1) END +
SUBSTRING(CONVERT(NVARCHAR,#OFDate,100),14,4) + ' ' + RIGHT(CONVERT(NVARCHAR,#OFDate,100),2)
which you can use to offset the current date to prove that it works for multiple cases. For instance, when I use the numbers 0, 1, 12 and 13 right now, I get:
04/18/2016 09:34 AM
04/18/2016 10:34 AM
04/18/2016 09:34 PM
04/18/2016 10:34 PM
which means you can probably guess my time zone.
This is pretty cumbersome code. I don't know if you can do any better or not, but it will hopefully get you started. I suggest that if you are going to be needing this in a lot of places, but without a whole lot of access in your procedure, that you could use a function to return it. If you're going to be doing it for lots of different lines in a table, though, you're better off just to put the unelegant, complicated code right into your procedure.
CAST and CONVERT (Transact-SQL)
SELECT CONVERT (VARCHAR, GETDATE(), 101) + ' ' + CONVERT (VARCHAR,CONVERT (TIME, GETDATE()))
or
change the language setting in your session with
SET LANGUAGE us_english
SELECT * FROM sys.syslanguages
With Microsoft Sql Server:
--
-- Create test case
--
DECLARE #myDateTime DATETIME
SET #myDateTime = '2016-04-03'
--
-- Convert string
--
SELECT LEFT(CONVERT(VARCHAR, #myDateTime, 120), 10)

How can I script SQL Server database diagrams?

How can I export SQL Server database diagrams as developer-friendly SQL scripts?
By developer-friendly, I mean written in a way similar to the way a human would write them as opposed to the messy many-UPDATEs style used by existing solutions.
(Note that similar questions on this site only seem to cover specific versions of SQL Server or migration of diagrams.)
Here's a script to do this. Tested in SQL Server 2008 R2 and 2012.
DECLARE #values nvarchar(max);
SET #values =
(
SELECT '
(''' + REPLACE(name, '''', '''''') + ''', ' + CAST(principal_id AS VARCHAR(100)) +', ' + CAST(version AS VARCHAR(100)) + ', ' + sys.fn_varbintohexstr(definition) + '),'
FROM sysdiagrams
FOR XML PATH(''), TYPE
).value('.', 'nvarchar(max)');
SET #values = LEFT(#values, LEN(#values) - 1);
SELECT
'IF OBJECT_ID(N''dbo.sysdiagrams'') IS NULL
CREATE TABLE dbo.sysdiagrams
(
name sysname NOT NULL,
principal_id int NOT NULL,
diagram_id int PRIMARY KEY IDENTITY,
version int,
definition varbinary(max)
CONSTRAINT UK_principal_name UNIQUE
(
principal_id,
name
)
);
MERGE sysdiagrams AS Target
USING
(
VALUES' + #values + '
) AS Source (name, principal_id, version, definition)
ON Target.name = Source.name
AND Target.principal_id = Source.principal_id
WHEN MATCHED THEN
UPDATE SET version = Source.version, definition = Source.definition
WHEN NOT MATCHED BY Target THEN
INSERT (name, principal_id, version, definition)
VALUES (name, principal_id, version, definition);
';
It basically exports the contents of the sysdiagrams table. Note that it does not retain the diagrams' id numbers. It also retains who created the diagrams, but the id number should also exist in the target database.
If you run the resultant script on a server instance that doesn't have the database diagramming objects, it should still work. However, after doing this, in order for them to appear in SSMS, I think you'll need to expand the Database Diagrams node and click Yes when asked to create them.
This is based on the 2008 script from here.
Note that there is a catch! SSMS and other Microsoft tools truncate the resulting text in the result set if you have more than a few diagrams. To get the full text, here's a PowerShell script to run the query and put the output in the clipboard:
$ErrorActionPreference = "Stop"
function Pause([string]$message) {
Write-Host $message
$host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown") | Out-Null
}
function Set-Clipboard {
$input | PowerShell -NoProfile -STA -Command {
Add-Type -AssemblyName "System.Windows.Forms"
[Windows.Forms.Clipboard]::SetText($input)
}
}
$connection = New-Object System.Data.SqlClient.SqlConnection ("Data Source=DATABASE_INSTANCE;Initial Catalog=DATABASE;Integrated Security=SSPI")
$connection.Open()
$command = $connection.CreateCommand()
$command.CommandText = #"
--SQL CODE
"#
$command.CommandTimeout = 60
$result = $command.ExecuteScalar()
$command.Dispose()
$connection.Dispose()
Pause "Press any key to copy the resulting SQL to the clipboard..."
$result | Set-Clipboard
Fill in the database, instance name, and SQL placeholders.
#Sam's answer is 100% correct and works (against 2019, too), and you still must do as he says: executing using Powershell
His original PS script features a Pause near the end, and that bugs-out for me when I run the script in Powershell ISE (probably due to my own naivete)
So, here's my slightly changed PS script (with the SQL embedded already) that is a near direct steal of #Sam's lovely work
$ErrorActionPreference = "Stop"
function Set-Clipboard {
$input | PowerShell -NoProfile -STA -Command {
Add-Type -AssemblyName "System.Windows.Forms"
[Windows.Forms.Clipboard]::SetText($input)
}
}
$connection = New-Object System.Data.SqlClient.SqlConnection ("Data Source=localhost;Initial Catalog=MySpecialDataBase;Integrated Security=SSPI")
$connection.Open()
$command = $connection.CreateCommand()
$command.CommandText = #"
DECLARE #values nvarchar(max);
SET #values =
(
SELECT '
(''' + REPLACE(name, '''', '''''') + ''', ' + CAST(principal_id AS VARCHAR(100)) +', ' + CAST(version AS VARCHAR(100)) + ', ' + sys.fn_varbintohexstr(definition) + '),'
FROM sysdiagrams
FOR XML PATH(''), TYPE
).value('.', 'nvarchar(max)');
SET #values = LEFT(#values, LEN(#values) - 1);
SELECT
'IF OBJECT_ID(N''dbo.sysdiagrams'') IS NULL
CREATE TABLE dbo.sysdiagrams
(
name sysname NOT NULL,
principal_id int NOT NULL,
diagram_id int PRIMARY KEY IDENTITY,
version int,
definition varbinary(max)
CONSTRAINT UK_principal_name UNIQUE
(
principal_id,
name
)
);
MERGE sysdiagrams AS Target
USING
(
VALUES' + #values + '
) AS Source (name, principal_id, version, definition)
ON Target.name = Source.name
AND Target.principal_id = Source.principal_id
WHEN MATCHED THEN
UPDATE SET version = Source.version, definition = Source.definition
WHEN NOT MATCHED BY Target THEN
INSERT (name, principal_id, version, definition)
VALUES (name, principal_id, version, definition);
';
"#
$command.CommandTimeout = 60
$result = $command.ExecuteScalar()
$command.Dispose()
$connection.Dispose()
$result | Set-Clipboard
echo "Your SQL Diagram was successfully scripted out and copied to the clipbaord"
Just launch your Windows PowerShell ISE, paste this into the top pane, update the connection string ~ line 10. Save the script. Run the script.

ssis expression how to convert datetime sql statement?

In ssis package I have String type variable V2 inside expression property i'm writing following sql query
"select * from mytable where date = " + #[System::StartTime]
But it is giving me an error :
The data types "DT_WSTR" and "DT_DATE" are incompatible for binary operator "+". The operand types could not be implicitly cast into compatible types for the operation. To perform this operation, one or both operands need to be explicitly cast with a cast operator.
Attempt to set the result type of binary operation ""select * from table where date = " + #[System::StartTime]" failed with error code 0xC0047080.
Even I have also tried with (DT_WSTR) #[System::StartTime]
still no luck any advice ?
You need to change data type of both StartTime variable and [date] field from the query to string.
Try this:
"select * from mytable where convert( varchar(10), [date], 120) = '" + SUBSTRING((DT_WSTR,50)#[System::StartTime],1, 10) + "'"
Which should return a proper query:
select * from mytable where convert( varchar(10), [date], 120) = '2013-05-22'
convert() will give you string like "2013-05-22". In my system (DT_WSTR) cast on #[System::StartTime] is returning string "2013-05-22 16:14:43", but if you have other settings, you'd have to construct the string from dateparts, if your default result would be for example "05/22/2013 16:14:43" due to other regional setting.
What is the verion of sql server you are using? and [date] field type exactly?

SQL Server equivalent of MySQL DATE_FORMAT()

Hope we're having a good day and all set for Christmas.
Got a quick question. I'm converting a MySQL function into SQL Server, I've got most of the function converted except for one part which has the following:
#description = CONCAT(#description, date_format(#INLastUpdated, '%H:%i %d %b %Y'))
What I'm trying to do is to basically recreate the date_format function to format the date in the same way specified, but I'm not sure how to do it. from what I've seen in the MySQL documentation the format selected would give hour:minute day / short month name / year.
Anyone got any ideas?
You can try this:
DECLARE #description VARCHAR(1000) = 'test'
DECLARE #INLastUpdated DATETIME = '2012-12-21 14:32:22'
SET #description = #description + ' '
+ LEFT(CONVERT(VARCHAR(8), #INLastUpdated, 8), 5) + ' '
+ CONVERT(VARCHAR(20), #INLastUpdated, 106)
SELECT #description
But be careful as format 106 depends on local language settings. Read more on MSDN
The equivalent function is CONVERT. But you're basically out of luck. SQL Server does not allow to cherry-pick the date tokens. You need to browse the available full date built-in formats and choose one, or try to compose an output by string concatenation, as in:
CONVERT(VARCHAR, CURRENT_TIMESTAMP, 103) + ' ' + CONVERT(VARCHAR(8), CURRENT_TIMESTAMP, 114)
Which version of SQL Server? In SQL Server 2012 we now have the FORMAT function.
This, in MySQL
date_format(#INLastUpdated, '%H:%i %d %b %Y')
translates into something like this for SQL Server 2012 using the FORMAT function:
DECLARE #d DATETIME = GETDATE();
SELECT FORMAT(#d, 'HH:mm d MMM yyyy', 'en-US') AS 'DateTime Result';
BOL SQL Server 2012 > FORMAT (Transact-SQL)

adding current date as a parameter to a query in clover ETL

I have a query block for retrieving data from MSSQL Server. the query has some hardcoded date values which needs to be changed everyday to import the daily feed. I need to automate this execution. I am using cloverETL for executing the query right now.
Here is the query (its a query to retrieve sharepoint activity data)
use
DocAve_AuditDB;
DECLARE
#ParameterValue VARCHAR(100),
#SQL
VARCHAR(MAX)
SET
#SQL = STUFF((SELECT 'UNION ALL SELECT COL_ItemTypeName, COL_UserName, COL_MachineIp, COL_DocLocation, DATEADD(SECOND, COL_Occurred / 1000, ''19700101 00:00'') as Date_Occurred, COL_EventAction FROM '+ TABLE_NAME + ' WHERE DATEADD(SECOND, COL_Occurred / 1000, ''19700101 00:00'') BETWEEN '+ '''20120515'''+ 'AND' + '''20120516'''+ 'AND ' + 'COL_ItemTypeName='+ '''Document''' AS 'data()'
FROM INFORMATION_SCHEMA.TABLES
WHERE
TABLE_NAME LIKE '%2012_05%'
FOR
XML PATH('')),1,10,'')
EXEC
(#SQL)
In the above block I want the TABLE_NAME LIKE param i.e. %2012_05% to be a variable retrieved from the current data and also the date values in the between clause
BETWEEN '+ '''20120515'''+ 'AND' + '''20120516'''
to be todays date-1 and todays date
should create a small java program for handling this or it can be done directly in the query itself? if yes how?
Thanks in Advance
Use GETDATE() or CURRENT_TIMESTAMP to obtain the current date (and time).
Use CONVERT() with the 112 format specifier to convert the current timestamp to a string formatted as YYYYMMDD.
Use DATEADD() for calculations (like subtracting one day) on dates/times.
Use SUBSTRING() to subtract parts from the formatted date string to rearrange them to the %YYYY_MM% format.
Or you can use inline ctl notation in DBInputTable:
SELECT 'UNION ALL SELECT COL_ItemTypeName, COL_UserName, COL_MachineIp, COL_DocLocation, DATEADD(SECOND, COL_Occurred / 1000, ''19700101 00:00'') as Date_Occurred, COL_EventAction FROM `date2str(today(), "yyyy_MM")` WHERE DATEADD(SECOND, COL_Occurred / 1000, ''19700101 00:00'') BETWEEN '+ '''`date2str(today(), "yyyyMMdd")`'''+ 'AND' + '''`date2str(dateAdd(today(),1,day), "yyyyMMdd")`'''+ 'AND ' + 'COL_ItemTypeName='+ '''Document''' AS 'data()'