When saving certain hexadecimal values to our database, they are being converted to what I think is scientific (E) notation. e.g. 558E74 becomes 5.6e76.
I can understand that the number 558E74 is also represented as 5.6e76, but the value is not intended to be a number, so the conversion is not what we're after!
We're currently using:
MySQL 5.7
Column type is char with a max length of 6
How can we prevent the values from being converted? Is there something we can change about the way they are saved? Or should we be using a different column type?
I feel like I might be missing something fairly obvious as this seems like quite a basic question (apologies if so!), but having searched around I was struggling to find any answers.
If its a character destination, quote it, otherwise hexadecimal literal like 0x558E74
Related
I'm searching for cases in MySQL/MariaDB where the value transmitted when storing will differ from the value that can be retrieved later on. I'm only interested in fields with non-binary string data types like VARCHAR and *TEXT.
I'd like to get a more comprehensive understanding on how much a stored value can be trusted. This would especially be interesting for cases where the output just lacks certain characters (like with the escape character example below) as this is specifically dangerous when validating.
So, this boils down to: Can you create an input string (and/or define an environment) where this doesn't output <value> in the second statement?
INSERT INTO t SET v = <value>, id = 1; // success
SELECT v FROM t WHERE id = 1;
Things I can think of:
strings containing escaping (\a → a)
truncated if too long
character encoding of the table not supporting the input
If something fails silently probably also depends on how strict the SQL mode is set (like with the last two examples).
Thanks a lot in advance for your input!
you can trust that all databases do, what the standards purpose, with strings and integer it is simple, because it saves the binary representation of that number or character in your choosen character set.
Decimal Double and single values are different, because the can't be saved directly and so it comes to fractals see decimal representation
That also follows standards, but you have to account with it.
I have a Python script which collects data and sends it to my MySQL table.
I noticed that the "Cost" sometimes is 0,95 which results in 0 in my table since my table use "0.95" instead of "0,95".
I assume the best solution is to convert the , to . in my Python script by using:
variable.replace(",", ".")
However, couldn't one solution be to change format in my MySQL table? So that I store numbers in this format:
1100
0,95
0,1
150000
My Django Model
cost = models.DecimalField(max_digits=10, decimal_places=4, default=None)
Any feedback on how to best solve this issue?
Thanks
Your first instinct is correct: convert the "unusual" (comma-decimal) input into the standard format that MySQL used by default (dot-decimal) at the first point where you receive it.
there's lots of ways to write numbers
Be careful, though that you don't get stung by people using commas as thousands separators like "3,203,907.23", or the European form "3.203.907,23", the Swiss "3'203'907,23' or even this form, which is widely used in India: "32,03,907.71" (yes, I did mean to type only two digits there!)
To make your life easier, the rule for currencies is relatively simple:
where a dot or comma is followed by only two digits at the end of the string, that character is acting as the decimal separator.
Once you know which is the decimal separator, you can safely remove all other non-digits from the string, change the decimal separator you found to . then use any standard library string-to-number conversion.
Storage format isn't presentation format
Yes, you can tell MySQL to use comma as its decimal separator, but doing that will break so much of your code - including the parts of the framework that read from the database and expect dot-decimal numbers - that you'll regret doing it that way very quickly...
There's a general principle at work here: you should do your data storage and processing using a format that is easy to process, interchangeable with other systems, and understood by other software developers.
Consider what happens if you need to allow a different framework to access your MySQL database to generate reports... whoever develops that software (and it may be you) will be glad that the numbers are all stored the way numbers are "always" stored in databases.
Convert on the way in, re-convert on the way out
Where you need to accept input in a different format, convert that input into your standardised format as early as possible.
When you need to use an output format, do the conversion to that format as late as possible.
The idea is to keep as much of your system "unexceptional" as possible. A programmer who has to remember what numeric format will in force at the time when a given method is called is not a happy programmer.
P.S.
The option you're talking about in MySQL is an example of this pattern: it doesn't change how numeric data is stored. All that changes is how you pass numbers to MySQL and how it presents them back to you.
I'm using MySQL, and I am trying to find common strings over a given character length within a series of messages that are highly dynamic, Each message may have a common phrase, but they will be appended with reference codes or names that don't match a specific format on either side of the string. for example, this is an example of the types of common phrases I'm trying to scan for, but has dynamic content embedded as well, and in different formats (https://screencast.com/t/rlABTWitQ)
The end result I am looking for is something akin to this (https://screencast.com/t/qXzrGNFuf)
Because of the highly variable nature of the formats of these messages, uses of substring_index and regexp (as much as my amateur familiarity with REGEXP has taken me), I can't seem to get anything going
SELECT LEFT("first_middle_last", CHAR_LENGTH("first_middle_last") - LOCATE('_', REVERSE("first_middle_last")));
I can't use something like this, as it would just strip out on a specific type of character. As you can see, the types of strings are too variant in format
I'm working on optimizing my parser for JSON that I built in Visual Basic .NET. I do not follow the EBNF verbatim, for example with numbers I match for an optional positive sign and for boolean/null values I'm not limiting a match to just lower-case. However, I have a question on if I should use Char.IsDigit or NumberFormat.NativeDigits for matching digits in a number.
Currently I use Char.IsDigit because I'm iterating through each character in the source code and so it is just easier to compare the currently iterated Char value in the String. However, to check for the optional positive/negative signs I'm using the NumberFormat class anyways and so I was wondering if there is any benefit to checking if the currently iterated character is in the NativeDigits collection.
The downside that I can think of is that since I am iterating through each Char in the String, that I'd have to convert the Char to a String to check if the character is in the NativeDigits collection; since Strings are immutable in VB.NET I try to make as few instances of Strings as possible.
In mysql, if I do something like
round((amount * '0.75'),2)
it seem to work just fine like without single quotes for 0.75. Is there a difference in how mysql process this?
In the hope to close out this question, here's a link that explains type conversion in expression evaluation: https://dev.mysql.com/doc/refman/5.5/en/type-conversion.html
When an operator is used with operands of different types, type
conversion occurs to make the operands compatible. Some conversions
occur implicitly. For example, MySQL automatically converts numbers to
strings as necessary, and vice versa.
mysql> SELECT 1+'1';
-> 2
In your case, MySQL sees arithmetic and performs implicit conversion on any string contained in the expression. There is going to be an overheard in converting a string to number, but it's negligible. My preference is to explicitly type out a number instead of quoting it. That method has helped me in code clarity and maintainability.