I have a column from my .csv file coming in with values as 1754625.24 etc,. where as we have to save it as integer in our database. So am trying to split number with '.' and divide second part with 1000 (24/1000) as i want 3 digit number.
so i get 0.024. But i am having issues storing/preserving that value as a decimal.
I tried (DT_DECIMAL,3) conversion but i get result as '0'.
My idea is to then append '024' part to original first part. So my final result should look like 1754625024
Please help
I am not convinced why would you store 1754625.24 as 1754625024 when storing it as int.
But still for your case , we can use a derived column task and
use Replace command on the source column of csv. E.g.
Replace('1754625.24','.',0)
Related
Please look at my screenshots and help me to understand what I am missing.
What datatype should I choose for these columns in MYSQL? I keep getting mistakes in decimal datatype columns. I chose decimаl12,3 because no columns(revenue, product&purchase price) with currency have more than 12 digits in total, 9 before and 3 after the decimal point. Could someone help me to understand what data type to choose with examples?
if we have an integer number e.g. 85192 we choose int?
for currency we choose the decimal, right? then what have I done wrong that I keep getting errors? 0 records imported.
if we have a combination of numbers and letters or just letters then we choose varchar? and varchаr1 equals 1 character, eg. apple32 = 7 characters, therefore vаrchar7?
turning to decimal, 12,464.87 in total 7 digits, 5 before and 2 after the decimal point, hence mysql decimаl7,2 should be enough, right? or would it be better to put decimаl10,3 with a margin so to say.
excel
mysql
data
$1,000.00 contains two characters that cannot be part of a numeric literal: the dollar sign and the comma that is used as a thousands separator.
Find a way to change '$1,000.00' to '1000.00' in the input file. Then, the load will succeed.
Alternatively, create an intermediate table where product_price is a VARCHAR(32), load into that, and then:
INSERT INTO target_table
SELECT
other_col1
,other_col2
, ....
,CAST(REPLACE(REPLACE(product_price,',',''),'$','') AS DECIMAL(15,2)
,other_col_n
,...
FROM staging_table;
You don't need an intermediate table. When doing LOAD DATA, put and columns into #variables; then use a SET to convert as needed:
LOAD DATA
...
col1, col2, #price, ...,
SET price = CAST(REPLACE(REPLACE(product_price,',',''),'$','') AS DECIMAL(15,2))
Dates need to be like this: "2022-07-25 22:02:22". Either change what Excel is delivering, or use STR_TO_DATE(...) in the SET.
I have to send data from csv into SQL DB.
Problem starts when I try to convert data into Int. It wasnt my idea and I really cant do much with this datatype. When I'm trying to achieve this problem pop up:
Data Conversion 2: Data conversion failed while converting column
"pr_czas" (387) to column "C pr_dCz_id" (14). The conversion returned
status value 2 and status text "The value could not be converted
because of a potential loss of data.".
Tried already to ignore this problem but then another problems came up so there is no other way than solving this.
I have to convert this data from csv file which is str 50 into int 4
It must be int4. One of the requirements Dont know what t odo.
This is data I'm trying to put into int4. Look on pr_czas
This is data's datatype
Before I tried to do same thing with just DD.MM.YYYY but got same result...
Given an input column named [pr_czas] that contain string values that look like 31.01.2020 00:00 which appears to be a formatted date time represented in the format "DD.mm.YYYY HH:MM", I would like to express that as a whole number DDMMYYHHMM
Add a derived column to your data flow and call this new_pr_czas
The logic I'm going to use is a series of REPLACE statements and cast the final result to an integer. Replace the period, replace the colon and the space - all with nothing
(DT_I8)REPLACE(REPLACE(REPLACE([pr_czas], ".", ""), ":", ""), " ", "")
This is an easy case but things to note.
An integer/int32/I4 has a maximum value of 2 billion.
310120200000 is too large to fit into that space so you would need to make that an bigint/int64/I8. If I remember your previous question, you were having troubles with a lookup task so this data type mismatch might hurt you there.
The other thing to be aware of is that leading zeros will be dropped when converted to a number because they are not significant. If you need to retain the leading zeros, then you're working with string data type. This is an advantage to working with the ISO standard but if your data expects DD, then far be it for me to say otherwise.
If you need to slice your date into another format, then you'll want to have a few derived columns. The first one will generate a string column for each piece of pr_czas - year, month, day, hour and minute. You'll use the substring method for this and findstring to find the period space and colon.
The next data flow will be used to put those string pieces back into the new format and cast that to I8. Why? Because you can't debug doing it all in one shot but you can put a data viewer between two derived columns to figure out where a slice went awry.
I am having some difficulties about storing integer data with commas, I have prices, which is like 4,600 So I need to store it with commas but when I try to send it as Integer it cut after first number. I tried to change column type. BigInt or Double but it doesn't effect any. Any possible way to do that?
Also tried to change comma to dot "." but with this, mysql delete the "0" at last... I don't know why...
Prices
------
4,500
2,300
1,500
Because you're using a comma, MySQL most likely interprets the number as two fields, separated by the comma. For example:
Prices,Unspecified
------,-----------
4 ,500
2 ,300
1 ,500
In the numbers in question: If the comma is a thousands separator, remove it (via String replace) before trying to store the number. If it's a decimal point, replace it with a period (via String replace) and store it as a DOUBLE (or DECIMAL if you need high accuracy for large numbers).
If you want to display the number with a comma, use String formatting (possibly a number-formatting function other than String.format() or sprintf()) after retrieving the value from the database.
If you want to be able to do calculations using SQL queries with those numbers, then you need to store the price either use the DECIMAL type or use integer types and multiply the number by e.g. 1000 before saving.
Double or any other floating point representation of numbers are not suitable for price calculations/storage.
If you use DECIMAL need to convert the number form your local format 4,5000 to the format the database expects when you store it in the database, and convert it back to the local format when you retrieve your data.
If you store it as string then you can keep your local format but that's the worst solution, and should never be used.
I'm using SSIS to separate good data from unusable date. In order to do that I used derived columns, script task and conditional split where I assigned certain conditions. One of the conditions I need to apply is that none of the numbers in one column cannot be negative. I'm guessing that the best way to solve this would be using conditional split, but I cannot get it to work. I'm new to SSIS, so any help would be appreciated.
You'd have an Expression like
[MyCaseSensitiveColumnName] < 0
and then name the output path something like BadData_NegativeValue
From the comments
that is what I did before, but I'm getting an error saying that The data types "DT_WSTR" and "DT_I4" are incompatible for binary operator ">"
That error message indicates that you are attempting to compare a unicode string (DT_WSTR) and an integer (DT_I4) and that the expression language does not allow it.
To resolve this type incompatibility, you would need to first convert the value of MyCaseSensitiveColumnName from DT_WSTR to an integer.
I'd likely add a Derived Column Component to my data flow and create a new column called MyCaseSensitiveColumnNameAsInteger with an expression like
(DT_I4) [MyCaseSensitiveColumnName]
Now, that may be perilous depending on the quality of your source data. I don't know why you are pulling numeric data in as a string. If there could be non whole numbers in the data set, then we will need to check before making the cast. If there are NULLs in that dataset, those too may cause issues.
That would result in our conditional split check becoming
[MyCaseSensitiveColumnNameAsInteger] < 0
I have numeric data with have value 0.546, 0.456 in the database. When I try to export these values to flat file using flat file manager the zero value is truncated and only .546,.456 is shown in the flat file.
The value other than zero doesn't have this problem.
I tried using data conversion but of no use
Try Data Conversion transform - Data Type: decimal [DT-DECIMAL) : Scale 3.
Good find, Prakash!
I'm afraid you'll have to first convert you numeric data to DT_WSTR. It has to be Derived Column transformation, not Data Conversion, because you'd get the same result. In expression you need to prepend converted number with 0. Don't know about negative numbers, but maybe they will have correct format (you need to test it).
floor(decimal_column)==0 ? "0" + (DT_WSTR,10)decimal_column : (DT_WSTR,10)decimal_column
I know that's not what you expected, but I had the same problem the other day and was unable to finde better solution :).