I just want to format a decimal number for output to a simple CSV formatted file.
I feel like I'm stupid, but I can't find a way to do it without leading zeroes or spaces, of course I can simply trim the leading spaces, but there has to be a proper way to just format like I that, isn't there?
Example
define variable test as decimal.
define variable testString as character.
test = 12.3456.
testString = string(test, '>>>>>9.99').
message '"' + testString + '"' view-as alert-box. /* " 12.35" */
I tried using >>>>>9.99 and zzzzz9.99 for the number format, but both format the string with leading spaces. I actually have no idea what the difference is between using > and z.
The SUBSTITUTE() function will do what you describe wanting:
define variable c as character no-undo.
c = substitute( "&1", 1.23 ).
display "[" + c + "]".
(Toss in a TRUNCATE( 1.2345, 2 ) if you really only want 2 decimal places.)
Actually, this also works:
string( truncate( 1.2345, 2 )).
If you are creating a CSV file you might want to think about using EXPORT. EXPORT format removes leading spaces and omits decorations like ",". The SUBSTITUTE() function basically uses EXPORT format to make its substitutions. The STRING() function uses EXPORT format when no other format is specified.
The EXPORT statement will format your data for you. Here is an example:
DEFINE VARIABLE test AS DECIMAL NO-UNDO.
DEFINE VARIABLE testRound AS DECIMAL NO-UNDO.
DEFINE VARIABLE testString AS CHARACTER NO-UNDO.
test = 12.3456.
testRound = ROUND(test, 2).
testString = STRING(test).
OUTPUT TO VALUE("test.csv").
EXPORT DELIMITER "," test testRound testString.
OUTPUT CLOSE.
Here is the output:
12.3456,12.35,"12.3456"
The EXPORT statement's default delimiter is a space so you have to specify a comma for your CSV file. Since the test and testRound variables are decimals, they are not in quotes in the output. testString is character so it is in quotes.
Related
I have below data with 5 variables with 5 observations: Name Age Gender Weight country
data have;
length string $30.;
input string$;
datalines;
Naresh30Male70India
Venkey29Male50Kenya
Ravi30Male56Pak
Sai67Female40iran
Divya89Female78Dubai
;
run;
I want to Separate these 5 variables in 5 observations
Help me on it
This is fixed code that should work for any range of ages/weights. Not the most elegant solution, but it works. But I would still rather fix that at the input site, and put some delimiters into the original string.
data want;
set have;
mixed2=substr(Mixeddata, anydigit(Mixeddata), anydigit(Mixeddata,-50)-anydigit(Mixeddata)+1);
mixed3=substr(mixed2, notdigit(mixed2));
name=substr(Mixeddata,1, anydigit(Mixeddata)-1);
age=substr(mixed2,1,notdigit(mixed2)-1);
gender=substr(mixed2, notdigit(mixed2), anydigit(mixed2, 5)-notdigit(mixed2));
country=substr(Mixeddata, anydigit(Mixeddata, -50)+1);
weight=substr(mixed3, anydigit(mixed3));
drop Mixeddata mixed2 mixed3;
run;
Consider the makeup of a single string:
Naresh30Male70India
Each desired column is either a number or a character. If we could break this out into two strings, one with only numbers and one with only characters, we can easily pull the needed values:
string_num: 30 70
string char: Naresh Male India
We can do this with regular expressions by replacing letters with spaces and numbers with spaces.
data want;
set have;
string_num = compbl(prxchange('s/[0-9]/ /', -1, string) );
string_char = compbl(prxchange('s/[a-zA-Z]/ /', -1, string) );
name = scan(string_num, 1);
age = scan(string_char, 1);
gender = scan(string_num, 2);
weight = scan(string_char, 2);
country = scan(string_num, 3);
drop string_num string_char;
run;
Note that we use the compbl function to remove any extra spaces to make it easier to read for learning purposes, but this is an optional step.
First week having to deal with a MYSQL database and JSON field types and I cannot seem to figure out why values are encoded automatically and then returned in encoded format.
Given the following SQL
-- create a multiline string with a tab example
SET #str ="Line One
Line 2 Tabbed out
Line 3";
-- encode it
SET #j = JSON_OBJECT("str", #str);
-- extract the value by name
SET #strOut = JSON_EXTRACT(#J, "$.str");
-- show the object and attribute value.
SELECT #j, #strOut;
You end up with what appears to be a full formed JSON object with a single attribute encoded.
#j = {"str": "Line One\n\tLine 2\tTabbed out\n\tLine 3"}
but using JSON_EXTRACT to get the attribute value I get the encoded version including outer quotes.
#strOut = "Line One\n\tLine 2\tTabbed out\n\tLine 3"
I would expect to get my original string with the \n \t all unescaped to the original values and no outer quotes. as such
Line One
Line 2 Tabbed out
Line 3
I can't seem to find any JSON_DECODE or JSON_UNESCAPE or similar functions.
I did find a JSON_ESCAPE() function but that appears to be used to manually build a JSON object structure in a string.
What am I missing to extract the values to the original format?
I like to use handy operator ->> for this.
It was introduced in MySQL 5.7.13, and basically combines JSON_EXTRACT() and JSON_UNQUOTE():
SET #strOut = #J ->> '$.str';
You are looking for the JSON_UNQUOTE function
SET #strOut = JSON_UNQUOTE( JSON_EXTRACT(#J, "$.str") );
The result of JSON_EXTRACT() is intentionally a JSON document, not a string.
A JSON document may be:
An object enclosed in { }
An array enclosed in [ ]
A scalar string value enclosed in " "
A scalar number or boolean value
A null — but this is not an SQL NULL, it's a JSON null. This leads to confusing cases because you can extract a JSON field whose JSON value is null, and yet in an SQL expression, this fails IS NULL tests, and it also fails to be equal to an SQL string 'null'. Because it's a JSON type, not a scalar type.
I am trying to read some data into julia into a data frame to work with it. A minimal example of the .csv file could look like this:
A; B; C; D
ab; 1,23; 4; 9,2
ab; 3,4; 7; 1,1
ba; 6; 2,3; 8,6
I load the following to packages and read the data:
using DataFrames
using CSV
d = CSV.read( "test.csv", delim=";")
Julia recognizes the following types:
eltypes(d)
CategoricalArrays.CategoricalString{UInt32}
String
String
String
How could I now turn whole columns to floats with the comma replaced by a dot? My first idea was to use:
float(d[1,2])
But I did not find an option to tell julia to replace the comma with a dot.
My next idea was to first replace the comma and then convert it:
float(replace(d[1,2], ",", "."))
That works fine on a single cell but not on a whole column:
float(replace(d[:,2], ",", "."))
MethodError: no method matching
replace(::WeakRefStrings.WeakRefStringArray{WeakRefString{UInt8},1,Union{}},
::String, ::String)
I also tried:
d = CSV.read( "test.csv", delim=";", decimal=",")
which also just gives an error ...
Any ideas how to handle this problem and how to efficiently read the data into julia?
Thanks a lot!
Best regards.
One straightforward way is to read the file to string, replace the comma decimal separators by dots and then create the DataFrame from it:
s = replace(readstring("test.csv"), ",", ".")
CSV.read(IOBuffer(s); delim=';', types=[String, Float64, Float64, Float64])
Note that you can use the types keyword to specifiy the column types (it will then implicitly parse the string entries).
EDIT: According to this github issue the CSV.jl's read method supports a decimal keyword (from version v0.2.0 on) which allows you to do
CSV.read("test.csv"; delim=';', decimal=',', types=[String, Float64, Float64, Float64])
EDIT: Removed hint to alternatively use readtable from DataFrames.jl because it seems to be deprecated in favor of CSV.read.
On a new project I work on I have data in CSV format to import into a mysql table. One of the columns is a price field which stores currency in the european format ie. 345,83.
The isssue I have is storing this decimal seperator. In most European currencies the decimal seperator is "," but when I try to insert a decimal number into a field (ex. 345,83), I get the following error: "Data truncated for column 'column_name' at row 'row #'". If I use '.' instead of ',' it works fine. Could you please help me with, how to store this format in mysql?
you can store it as a regular decimal field in the database, and format the number european style when you display it
edit: just added an example of how it might be achieved
$european_numbers = array('123.345,78', '123 456,78', ',78');
foreach($european_numbers as $number) {
echo "$number was converted to ".convert_european_to_decimal($number)."\n";
// save in database now
}
function convert_european_to_decimal($number) {
// i am sure there are better was of doing this, but this is nice and simple example
$number = str_replace('.', '', $number); // remove fullstop
$number = str_replace(' ', '', $number); // remove spaces
$number = str_replace(',', '.', $number); // change comma to fullstop
return $number;
}
Use number_format or money_format, it's pretty much what you preffer.
It's worse than you think. The number 1234.56 may be written in Europe as:
1234,56
1 234,56 (space as a group separator)
1.234,56 (dot as a group separator)
In .net the number parser can works according to a given culture, so if you know the format it does the hard work for you. I'm sure you can find a PHP equivalent, it'd save you a lot of trouble.
You could import the currency field into a VARCHAR column and then copy this column into a DECIMAL column while replacing the , by a . in all rows using MySQL string-manipulation-functions.
UPDATE <<table>>
SET <<decimal-currency-col>> = REPLACE(<<varchar-currency-col>>, ',', '.');
Some data types do not have a direct
correlation between SQL Server or
Access and MySQL. One example would be
the CURRENCY data type: MySQL does not
(yet) have a CURRENCY data type, but
creating a column with the definition
DECIMAL(19,4) serves the same purpose.
While MSSQL defaults to Unicode
character types such as nCHAR and
nVARCHAR, MySQL does not so tightly
bind character sets to field types,
instead allowing for one set of
character types which can be bound to
any number of character sets,
including Unicode.
from http://dev.mysql.com/tech-resources/articles/migrating-from-microsoft.html
You could also consider multiplying it by 100 and storing it as INT.
Before inserting the price to the DB:
$price = (int)$price*100;
After receiving price from the DB:
$price = number_format($price, 2, ',', ' ');
Try replacing the "," with "."?
$price = str_replace(",", ".", $price);
Am using opencsv 2.3 and it does not appear to be dealing with escape characters as I expect. I need to be able to handle an escaped separator in a CSV file that does not use quoting characters.
Sample test code:
CSVReader reader = new CSVReader(new FileReader("D:/Temp/test.csv"), ',', '"', '\\');
String[] nextLine;
while ((nextLine = reader.readNext()) != null) {
for (String string : nextLine) {
System.out.println("Field [" + string + "].");
}
}
and the csv file:
first field,second\,field
and the output:
Field [first field].
Field [second].
Field [field].
Note that if I change the csv to
first field,"second\,field"
then I get the output I am after:
Field [first field].
Field [second,field].
However, in my case I do not have the option of modifying the source CSV.
Unfortunately it looks like opencsv does not support escaping of separator characters unless they're in quotes. The following method (taken from opencsv's source) is called when an escape character is encountered.
protected boolean isNextCharacterEscapable(String nextLine, boolean inQuotes, int i) {
return inQuotes // we are in quotes, therefore there can be escaped quotes in here.
&& nextLine.length() > (i + 1) // there is indeed another character to check.
&& (nextLine.charAt(i + 1) == quotechar || nextLine.charAt(i + 1) == this.escape);
}
As you can see, this method only returns true if the character following the escape character is a quote character or another escape character. You could patch the library to this, but in its current form, it won't let you do what you're trying to do.