I'm trying to create a generic script to create a database and want to pass the database name as a variable; but it's not working.
Here is my code so far
DECLARE #ID sysname;
SET #ID = 'test'
SET #ID = QUOTENAME(#ID)
CREATE DATABASE #ID
ON PRIMARY
( NAME = [' + #ID + '], FILENAME = N'C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA\' +#ID+ '.mdf' , SIZE = 211968KB ,
MAXSIZE = UNLIMITED, FILEGROWTH = 1024KB )
LOG ON
( NAME = [' + #ID + '_log'], FILENAME = N'C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA\'#ID + '_log.ldf' , SIZE = 149696KB , MAXSIZE = 2048GB , FILEGROWTH = 10%)
which gives me the following error:
Incorrect syntax near '#ID'.
If you can use SQLCMD mode in SSMS (SQLCMD Mode in the Query menu) or execute your script with sqlcmd.exe, you can use a SQLCMD scripting variable:
:setvar ID "test"
CREATE DATABASE [$(ID)]
ON PRIMARY ( NAME = ['$(ID)_Primary'], FILENAME = N'C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA\$(ID).mdf' , SIZE = 211968KB , MAXSIZE = UNLIMITED, FILEGROWTH = 1024KB ) LOG ON ( NAME = ['$(ID)_log'], FILENAME = N'C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA\$(ID)_log.ldf' , SIZE = 149696KB , MAXSIZE = 2048GB , FILEGROWTH = 10%)
From the command line, you would need to comment out the :setvar ID "test" statement (i.e. --:setvar ID "test") and run the script like so...
sqlcmd.exe -E -i YourScript.sql -v ID="real"
...assuming Windows authentication, a local, default instance of SQL Server, and YourScript.sql in the current directory.
The key is the -v command-line option to specify the name of the SQLCMD variable you wish to set and its value.
Using a SQLCMD variable like this is essentially what SSDT does under the hood.
Related
I got a data frame in R querying a SQL Server DB, Now I want to loop on each line and insert it to MySQL DB
Tried with dbwritetable but it didn't work
library(RODBC)
library(odbc)
library(RMySQL)
con <- dbConnect(odbc(),
Driver = "SQL Server",
Server = "XX",
Database = "XX",
UID = "XX",
PWD = "XX",
Port = XX)
mydb = dbConnect(MySQL(), user='XX', password='XX', dbname='YY', host='YYY')
resultset <- dbGetQuery(con, "SET NOCOUNT ON
DECLARE #StartDate DateTime
DECLARE #EndDate DateTime
SET #StartDate = dateadd(d,-1,getdate())
SET #EndDate = getdate()
SET NOCOUNT OFF
SELECT …..
LEFT JOIN ... ON ….
LEFT JOIN …. ON x.Key = y.Key
WHERE temp.StartDateTime >= #StartDate")
nrows <- nrow(resultset)
colnames(resultset) <- c("tagName", "date_inserted","value") `
So in here I got my result, in resultset but I don't know how to insert the resulset in MySQL
dbWriteTable(mydb, name='data', value=resultset[0,],append=TRUE)
dbReadTable(mydb, "data")
I Expect to insert the data, but I don't know should it be a for loop (for each line a query) or how is it done
More details with this images :
This is my data set
This is MySQL DB structure
Try using a parameterized insert using the RODBCext package. I have used the following function in the past.
This will append records into your database
library(RODBC)
library(RODBCext)
First we need to connect to the database using RODBC.
sql.driver = "MySQL ODBC 5.3 ANSI Driver" # need to figure the version out
sql.server = "your_server_here"
sql.port = "3306" # or whatever your port number is
sql.user = "your_user_name_here"
sql.pass = "your_password_name_here"
sql.db = "your_database_name_here"
con.string = paste0("Driver=", sql.driver, ";",
"Server=", sql.server, ";",
"Port=", sql.port, ";",
"Uid=", sql.user, ";",
"Pwd=", sql.pass, ";",
"Database=", sql.db, ";")
ch = odbcDriverConnect(con.string)
Then here is the custom function saveTable(). You will want to run this with your specific inputs, defined below.
saveTable <- function(con, table_name, df) {
# con = the ODBC connection (e.g., ch)
# table_name = the SQL database table to append to
# df = the data.frame() to append
sql_code = paste("INSERT INTO",table_name,"(",paste(colnames(df),collapse=", "),") VALUES (",paste(rep("?",ncol(df)),collapse=","),")")
sqlExecute(con, sql_code, df)
}
I am writing a python script which connects to the SQL database. It creates databases based on the folder paths and tables based on the files present in those folders in corresponding databases.
My following code is doing everything fine but I want to optimize it in a way that it loads the data into tables only if the tables are empty.
The problem with the following code is that whenever I run it, it checks if the table is not present, it creates the table and if the table is already there, it moves on but when it comes to loading data into tables, every time run the script, it loads the data into tables from file 1.
I want to tweak it in a way that it loads data into tables only and only if the data is already not present in it. If the data is present, the code move on.
I tried to do something like create table if not exists but not successful.
hostname = 'hostname'
username = 'usrname'
password = '12345'
database = 'd1'
portname = '12345'
from mysql.connector.constants import ClientFlag
import pathlib
import sys
import os
import mysql.connector
import subprocess
from subprocess import *
import time
from termcolor import colored
print(colored('\nConnecting SQL database using host = '+hostname+' , username = '+username+' , port = ' +portname+ ' , database = ' +database+'.','cyan',attrs=['reverse','blink']))
print('\n')
myConnection = mysql.connector.connect(user=username, passwd=password,host=hostname,port=portname,database=database,client_flags=[ClientFlag.LOCAL_FILES])
myCursor =myConnection.cursor()
rootDir35 = '/mnt/Wdrive/pc35/SK/E13'
filenames35 = os.listdir(rootDir35)
root35 = pathlib.Path(rootDir35)
non_empty_dirs35 = {str(p35.parent) for p35 in root35.rglob('*') if p35.is_file()}
#35
try:
print(colored('**** Starting Performing SQL-Queries for pc35 **** \n','green',attrs=['reverse','blink'] ))
for f35 in non_empty_dirs35:
dB35 = f35.replace('/','_') or f35.replace('-','_')
for dirName35, subdirList35, fileList35 in os.walk(rootDir35):
if dirName35 == f35:
p1135= 'Current Working Directory is: %s' %f35+ ' '
print(colored(p1135,'cyan'))
createDB35='CREATE DATABASE IF NOT EXISTS %s' %dB35
myCursor.execute(createDB35)
p135='Database of pc35 Created : %s' %dB35
print(colored(p135,'cyan'))
useDB35='use %s' %dB35
myCursor.execute(useDB35)
myConnection.commit()
p235= 'Database in use : %s' %dB35
print(colored(p235,'cyan'))
print(' ')
for fname35 in fileList35:
completePath35 ='%s' %dirName35+'/%s'%fname35
tblname35 = os.path.basename(fname35).split('.')[0]
if '-' not in tblname35:
if '.' not in tblname35:
sql35= 'CREATE TABLE if not exists %s (Datum varchar(50), Uhrzeit varchar(13), UpsACT_V varchar(6), UpsPRE_V varchar(6), IpsACT_A varchar(6), IpsPRE_A varchar(6), PpsACT_W varchar(6), PpsPRE_W varchar(10), UelACT_V varchar(6), UelPRE_V varchar(6), IelACT_A varchar(8), IelPRE_A varchar(8), PelACT_W varchar(8), PelPRE_W varchar(8), Qlad_Ah varchar(10), Qlast_Ah varchar(10))' %(tblname35)
myCursor.execute(sql35)
myConnection.commit()
test35 = 'The Table %s ' %tblname35+ 'in database %s '%dB35+'is created'
print(colored(test35,'yellow'))
loadData35= "LOAD DATA LOCAL INFILE '%s' " %completePath35 + "INTO TABLE %s" %tblname35
myCursor.execute(loadData35)
myConnection.commit()
p335='Data loaded from file %s ' %fname35
p435=' into table %s ' %tblname35
p535 = p335 + p435
print(colored(p535,'green'))
print(' ')
print(colored('**** SQL-Queries for pc35 successfully executed **** \n','green',attrs=['reverse','blink']))
except:
print(' ')
print(colored('**** SQL queries for pc35 doesnot executed. Please refer to the report or user manual for more details ****','red',attrs=['reverse','blink']))
print(' ')
What I want is something like load data if not exists in table.
what do you think, is it possible or what should I do to achieve this?
I have created a table with 10 columns in matlab database toolbox. How do i retrieve the values and show them in matlab GUI? when I use the code below, I get Undefined function 'fetch' for input arguments of type 'double' error. Can anybody tell me what is wrong with my code?
f = getappdata(0,'fvalue');
%Set preferences with setdbprefs.
setdbprefs('DataReturnFormat', 'cellarray');
setdbprefs('NullStringRead', 'null');
%Make connection to database.
conn = database('marine_invertebrates', '', '');
%Read data from database.
curs = fetch(conn, sprintf(['SELECT description.commonName'...
' , description.scientificName'...
' , description.kingdom'...
' , description.phylum'...
' , description.subphylum'...
' , description.class'...
' , description.order'...
' , description.family'...
' , description.genus'...
' , description.species'...
' FROM marine_cbir.description WHERE description.imageID = "%s"'], num2str(f)));
curs = fetch(curs);
close(curs);
%Assign data to output variable
results = curs.Data;
commonName = set(handles.edit11,'String');
display (commonName);
scientificName = set(handles.edit1,'String');
display (scientificName);
kingdom = set(handles.edit2,'String');
display (kingdom);
phylum = set(handles.edit3,'String');
display (phylum);
subphylum = set(handles.edit4,'String');
display (subphylum);
class = set(handles.edit5,'String');
display (class);
order = set(handles.edit6,'String');
display (order);
family = set(handles.edit7,'String');
display (family );
genus = set(handles.edit8,'String');
display (genus);
species = set(handles.edit9,'String');
display (species );
Lets say Job "Alphabet" does tasks A-Z. In 15min mark the job will be in task M or in other words it will not complete in 15 min. . During my tests, I ran without a schedule or a one time run, it runs and completes successfully. Then I ran with a scheduler with " Everyday: every 15m". Here with a scheduler, I see the job never hits Z or never completes. Is the SQL agent stopping the instance and starting a new one ?
This is an easy one to test for yourself. In the following, I create a SQL Agent Job with a single step which creates a table in tempdb if it doesn't exist.
It then waits 90 seconds before inserting the current timestamp. But, it is scheduled to run every minute.
USE [msdb]
GO
DECLARE #jobId binary(16)
EXEC msdb.dbo.sp_add_job
#job_name = N'Overlapping Execution'
, #enabled = 1
, #notify_level_eventlog = 0
, #notify_level_email = 2
, #notify_level_netsend = 2
, #notify_level_page = 2
, #delete_level = 0
, #category_name = N'[Uncategorized (Local)]'
, #job_id = #jobId OUTPUT
SELECT
#jobId
GO
EXEC msdb.dbo.sp_add_jobserver
#job_name = N'Overlapping Execution'
, #server_name = N'localhost\DEV2014'
GO
USE [msdb]
GO
EXEC msdb.dbo.sp_add_jobstep
#job_name = N'Overlapping Execution'
, #step_name = N'Insert into table'
, #step_id = 1
, #cmdexec_success_code = 0
, #on_success_action = 1
, #on_fail_action = 2
, #retry_attempts = 0
, #retry_interval = 0
, #os_run_priority = 0
, #subsystem = N'TSQL'
, #command = N'USE tempdb
GO
IF NOT EXISTS
(
SELECT * FROM sys.tables AS T WHERE T.name = ''WatchMe''
)
BEGIN
CREATE TABLE dbo.Watchme
(
StartTime datetime NOT NULL
);
END
GO
-- wait for 90 seconds to ensure overlap
WAITFOR DELAY ''00:01:30'';
-- Add a row so we can demonstrate activity
INSERT INTO
dbo.Watchme
(
StartTime
)
VALUES
(CURRENT_TIMESTAMP);
'
, #database_name = N'tempdb'
, #flags = 0
GO
USE [msdb]
GO
EXEC msdb.dbo.sp_update_job
#job_name = N'Overlapping Execution'
, #enabled = 1
, #start_step_id = 1
, #notify_level_eventlog = 0
, #notify_level_email = 2
, #notify_level_netsend = 2
, #notify_level_page = 2
, #delete_level = 0
, #description = N''
, #category_name = N'[Uncategorized (Local)]'
, #notify_email_operator_name = N''
, #notify_netsend_operator_name = N''
, #notify_page_operator_name = N''
GO
USE [msdb]
GO
DECLARE #schedule_id int
EXEC msdb.dbo.sp_add_jobschedule
#job_name = N'Overlapping Execution'
, #name = N'EveryMinute'
, #enabled = 1
, #freq_type = 4
, #freq_interval = 1
, #freq_subday_type = 4
, #freq_subday_interval = 1
, #freq_relative_interval = 0
, #freq_recurrence_factor = 1
, #active_start_date = 20141023
, #active_end_date = 99991231
, #active_start_time = 0
, #active_end_time = 235959
, #schedule_id = #schedule_id OUTPUT
SELECT
#schedule_id
GO
So, what happens? SQL Agent won't start the job if it's already running. If you try to manually run it
Start failed for Job 'Overlapping Execution'.
Request to run job Overlapping Execution (from User pity\dafool) refused because the job is already running from a request by User mr\T. (Microsoft SQL Server, Error: 22022)
Instead, the agent will skip the missed starts until it's able to actually start. Here you can see the history. It started at 4:50. Missed the start at 4:51 as it was already running but caught the 4:52 window.
If I query my watchme table
SELECT
WM.*
FROM
dbo.WatchMe AS WM
ORDER BY
1
I can see that yes, my insert times are approximately 90 seconds after the job would have started.
StartTime
2014-10-23 16:51:30.277
2014-10-23 16:53:30.767
2014-10-23 16:55:30.790
2014-10-23 16:57:30.793
2014-10-23 16:59:30.870
Directly answering the question
No, the Agent won't stop a currently executing job to start a new instance of it.
Turn on logging within your packages (I favor SQL Server and logging OnPre/PostExecute, OnError, OnTaskFailed) and you should be able to divine where it's at in the process along with any failure information.
Add a AA step where you check if the job is already running.
SELECT sj.name FROM msdb.dbo.sysjobactivity sja
INNER JOIN msdb.dbo.sysjobs sj ON sja.job_id = sj.job_id
WHERE sja.start_execution_date IS NOT NULL
AND sja.stop_execution_date IS NULL
AND name = 'Alphabet'
If so, skip steps A-Z.
I have made database in SQL SERVER 2008 and have enabled the filestream technique. Now, how can I save the image to the SQL Server from vb 6.0. My database query for filesstream is as follows:
CREATE DATABASE Photo;
GO
ALTER DATABASE Photo
ADD FILEGROUP PhotoDBFS CONTAINS FILESTREAM;
GO
DECLARE #FilePath varchar(MAX) = (SELECT SUBSTRING(physical_name, 1, CHARINDEX(N'master.mdf', LOWER(physical_name)) - 1)
FROM master.sys.master_files
WHERE database_id = 1 AND file_id = 1) + 'PhotoDBFS_Filestream';
DECLARE #SQL varchar(MAX) = '
ALTER DATABASE Photo ADD FILE (
NAME = PhotoDBFSFile,
FILENAME = ''' + #FilePath + ''')
TO FILEGROUP PhotoDBFS;';
EXECUTE(#SQL);
GO
My table structure where I want to save the image is as follows:
CREATE TABLE Photos
(
PhotoId bigint NOT NULL PRIMARY KEY IDENTITY(1,1),
Title varchar(100) NOT NULL DEFAULT(''),
Subject varchar(100) NOT NULL DEFAULT('No Subject'),
Place varchar(100) NOT NULL DEFAULT('Unknown'),
Comment varchar(1000) NULL DEFAULT(''),
Rating tinyint NOT NULL DEFAULT(0),
PhotoFile varbinary(MAX) FILESTREAM NOT NULL,
CONSTRAINT PhotoRatingCheck CHECK(Rating >=0 AND Rating<=10)
);
My vb code for saving the image is as follows:
'Saving in the table [Photos]
Set Rs = New Recordset
SqlString = "SELECT * FROM Photos"
Rs.Open SqlString, con, adOpenStatic, adLockOptimistic, adCmdText
Rs.AddNew
Rs("UniqueId") = txtUniqueId.Text
Rs("CategoryId") = tempCategoryId
Rs("AlbumId") = tempAlbumId
Rs("Title") = txtTitle.Text
Rs("Subject") = txtSubject.Text
Rs("Place") = txtPlace.Text
Rs("Comment") = txtComment.Text
Rs("Rating") = sliderRating.Value
Rs("PhotoFile") = PhotoLocation
Rs.Close
Set Rs = Nothing
I get this error while saving.
Multiple-step OLE DB operation generated errors. check each OLE DB status value, if available. No work was done.
I also want to read the saved image from the database. Any suggestion???
You can't update IDENTITY columns, so remove assignment on UniqueId field, this fails now for certain.
When accessing the BLOB column PhotoFile sometimes helps to be explicit using Value property of the Field object i.e. Rs!PhotoFile.Value = binaryData or Rs("PhotoFile").Value = binaryData
For a table to have one or more FILESTREAM columns, it must also have a column of the uniqueidentifier data type that has the ROWGUIDCOL attribute. Does the CREATE TABLE statement you pasted in the question even work?