I am trying to bulk import a .CSV file with 13 columns into SQL Server using Management Studio 2014 with the following script and am encountering truncation issues:
CREATE TABLE #BudgetPlan
(
BUSINESS_UNIT VARCHAR(50),
DEPT VARCHAR(50),
JOBCODE VARCHAR(50),
FISCAL_YEAR VARCHAR(4),
FISCAL_MN VARCHAR(3),
REG_HOURS DECIMAL(10, 2),
OT_HOURS DECIMAL(10, 2),
CONTRACT_HOURS DECIMAL(10, 2),
NON_PROD_HOURS DECIMAL(10, 2),
REG_SAL DECIMAL(10, 2),
OT_SAL DECIMAL(10, 2),
CONTRACT_DOL DECIMAL(10, 2),
NONPROD_SAL DECIMAL(10, 2)
)
BULK INSERT #BudgetPlan
FROM '\\nt115\TESTING\Consolidated Hours Dollars.csv'
WITH ( FIELDTERMINATOR = ',')
I get some errors:
Msg 4863, Level 16, State 1, Line 49
Bulk load data conversion error (truncation) for row 1, column 4 (FISCAL_YEAR).
Msg 4863, Level 16, State 1, Line 49
Bulk load data conversion error (truncation) for row 2, column 4 (FISCAL_YEAR).
Column 3 (JOBCODE) does have some commas within the string, however where there are rows with commas in this column they are surrounded by double quotes (") but I think is still causing an issue. Rows 1 and 2 below have commas within the column and are surrounded by double quotes, however row 3 does not have a comma within the JOBCODE column, and therefore does not contain double quotes.
Here is a sample of the CSV input file:
11000 - General Hospital,600 - Nursing Admin,"1018 - VP, Nursing Services",FY20,Jul,157,0,0,27,14045,0,0,2440
11000 - General Hospital,600 - Nursing Admin,"1018 - VP, Nursing Services",FY20,Aug,159,0,0,17,14223,0,0,1545
11000 - General Hospital,600 - Nursing Admin,3029 - Clinical Nurse Educator,FY20,May,0,0,0,0,0,0,0,0
Any help is appreciated, thanks!
Related
Mysql offers a UUID() function,
which returns an rfc 4122 version 1 guid.
This is an easily guessed timestamp + node_id bit string.
How may we insert randomized version 4 guids?
(Defining a new function requires permissions and is out-of-scope.
Ben Johnson offers an expression that is very nice but a little verbose.)
This inserts a version-4 random string, without dashes.
In the interest of conciseness it uses a slightly reduced portion of the key space,
just 120 bits.
(So 8-bits are predictable, they're constant.)
-- Produces version 4 guids for mysql, as its UUID() only offers version 1.
--
-- See https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random)
-- We consider variant 1 only, ignoring the Microsoft proprietary variant 2.
-- Version 1 is predictable timestamp.
-- Version 4 is 122 random bits + 6 constant bits.
--
-- The nil guid comes out like this:
-- UUID('00000000-0000-4000-8000-000000000000') # 8-4-4-4-12
-- The nybble '4' is constant version.
-- The nybble '8' has hi bit set, next bit cleared, plus two wasted bits.
-- We deliberately choose to emit just 120 random bits, for simplicity.
-- The RAND() function returns about 53 bits of entropy in the mantissa,
-- so for 15 nybbles we call it twice to obtain 106 ( > 60 ) unguessable bits.
-- The standard spelling of a guid, with four '-' dashes, is 36 characters.
-- We emit 32 hex characters, sans dashes.
INSERT INTO guid_test (guid) VALUES (
concat(substr(sha2(rand(), 256), 1, 12),
'4', substr(sha2(rand(), 256), 1, 3),
'8', substr(sha2(concat(rand(), rand()), 256), 1, 15)
)
);
I have details, subtotals and totals.
When I put avg function in totals line I have avg of every row.
I need avg of subtotals
How to do it?
week1
day1..... 2
day3..... 3
day4..... 4
day6..... 2
total.... 11 sum()
week2
day1..... 3
day2..... 2
total..... 5 sum()
Total
........... 16 sum() OK
............ 2,66666 avg() here should be (11+5)/2 =8
Result after implementing solution
I created a dataset to replicate your sample data as follows:
DECLARE #t TABLE (week int, day int, amount int)
INSERT INTO #t VALUES
(1, 1, 2),
(1, 3, 3),
(1, 4, 4),
(1, 6, 2),
(2, 1, 3),
(2, 2, 2)
SELECT * FROM #t
I then built a simple tablix as you had done (more or less)
I included the incorrect results you had for illustration and then added a new expression to calculate this correctly.
The result looks like this
You can ignore the other datasets, this is just a report I use for testing. Only dataset3 is used here.
The expression used was this
=Sum(Fields!amount.Value) / CountDistinct(Fields!week.Value)
You'll just need to edit this to match your field names. It basically just sums all the detail amounts then divides by the number of distinct weeks in the dataset.
I wanted to create an table
create table Oceny_projekty
(
OProj_Id int not null comment '',
ID_Projektu int comment '',
ID_Studenta int comment '',
OProj_Ocena decimal(1,1) comment '',
OProj_Data date comment '',
primary key (OProj_Id)
);
And fill it with sample data generated by PowerDesigner
insert into Oceny_projekty (OProj_Id, ID_Projektu, ID_Studenta, OProj_Ocena, OProj_Data)
values (2, 18, 10, '2.5', '1857-12-25');
And I've got this:
insert into Oceny_projekty (OProj_Id, ID_Projektu, ID_Studenta,
OProj_Ocena, OProj_Data) values (2, 18, 10, '2.5', '1857-12-25') Error
Code: 1264. Out of range value for column 'OProj_Ocena' at row 1
How can I modify command that can accept decimal numbers? (with integer numbers there is no problem)
Using MySQL Workbench 6.3, PowerDesigner 16.6
Thanks in advance!
Declaring OProj_Ocena as decimal(1,1) means that it has 0 integer digits (precision - scale = 1 - 1 = 0) and 1 fractional digit. It can only store values 0, 0.1, 0.2 ... 0.9.
The datatype you need is probably decimal(2,1).
Putting Marek Grzenkowicz's answer in another way, DECIMAL(M, D):
M = D + (the number of whole number digits). See 12.25.2 DECIMAL Data Type Characteristics
So, determine the number of whole number digits you want, those to the left of the decimal point, and the number of decimal places (D) that you need and add them to get M. IMO, they should have used the phrase "whole number digits" or "integer digits" to make this a bit more obvious even though what they say means that as M is for ALL digits. I misinterpreted this at first until I reread their description a few times.
I am wondering how to write a SQL statement for SQL Server 2008 that selects entries where a column contains a comma-delimited value (usually - there could only be one entry (and no leading comma)) for instance:
Column1
---------------------------
10-15,20-30,31-97,104-187
Values in the column represents comma delimited ranges.
In this case I want to find 25.
Please consider normalizing your database. You are adding insult to injury by using a single column to keep multiple ranges data. A normalized database would have another table for the ranges, with a start value and an end value, along with a foreign key to the original table.
Something like this:
CREATE TABLE tblRange
(
Range_itemId int, --(fk to the original table)
Range_From int,
Range_To int
)
Should you do this, your query would be simply:
SELECT i.*
FROM tblItems
INNER JOIN tblRange ON(Item_Id = Range_ItemId)
WHERE Range_From <= 25
AND Range_To >= 25
However, if you can't normalize your database then you would have to use a split string function to create rows from your comma delimited column, and then parse the text of each row to find what is the range of each row.
Here is an example:
create demo table
CREATE TABLE ZZZItems
(
ItemId int identity(1,1),
ItemRanges varchar(500)
)
populate demo table
INSERT INTO ZZZItems VALUES
('10 - 20, 20 - 30, 30 - 40'),
('40 - 50, 50 - 60, 60 - 70'),
('70 - 80, 80 - 90, 90 - 100'),
('10 - 20, 20 - 30, 30 - 40')
using CROSS APPLY to a split function table and extracting the ranges from the splitted data
;WITH CTE AS
(
SELECT ItemId,
CAST(LEFT(Data, CHARINDEX('-', Data) - 1) As Int) As RangeFrom,
CAST(RIGHT(Data, LEN(Data) - CHARINDEX('-', Data)) As Int) As RangeTo
FROM ZZZItems
CROSS APPLY dbo.Split(ItemRanges, ',')
)
-- select the item ids where the requested number fits in the range.
SELECT ItemId
FROM CTE
WHERE RangeFrom < 25
AND RangeTo > 25
clean up
DROP TABLE ZZZItems
Results:
ItemId
-----------
1
4
I didn't add the split function, you should choose your own from the article I've linked to.
1) Can I do a BulkInsert from a CSV file, into a table, such that the table has an identity column that is not in the CSV, and gets automatically assigned?
2) Is there any rule that says the table that I'm BulkInsert'ing into, has to have the same columns in the same order as the flat file being read?
This is what I'm trying to do. Too many fields to include everything...
BULK INSERT ServicerStageACS
FROM 'C:\POC\DataFiles\ACSDemo1.csv'
WITH (FORMATFILE = 'C:\POC\DataFiles\ACSDemo1.Fmt');
GO
SELECT * FROM ServicerStageACS;
Error:
Msg 4864, Level 16, State 1, Line 3 Bulk load data conversion error
(type mismatch or invalid character for the specified codepage) for
row 1, column 1 (rowID).
I'm pretty sure the error is because I have an identity.
FMT starts like this:
9.0
4
1 SQLCHAR 0 7 "," 1 Month SQL_Latin1_General_CP1_CI_AS
2 SQLCHAR 0 100 "," 2 Client SQL_Latin1_General_CP1_CI_AS
A co-worker recommended that it was easier to do the bulk insert into a view. The view does not contain the identity field, or any other field not to be loaded.
truncate table ServicerStageACS
go
BULK INSERT VW_BulkInsert_ServicerStageACS
FROM 'C:\POC\DataFiles\ACSDemo1.csv'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
GO
SELECT * FROM ServicerStageACS;