Im looking for a way to decode a string encrypted in base32 back to original string in mysql.
I know there is a SP to do this with base64 but cannot find anything for base32.
Is it possible? is there a stored procedure I can use somewhere?
What are ways to implement this?
Thanks!
BASE 64 or BASE 32 are not encrypted, they are just encoded. MySQL does not have a native function to perform encoding/decoding of Base 32 strings as it has for Base 64, FROM_BASE_64 e TO_BASE_64.
As an alternative you can try the CONV mathematical function (depending on the content stored as BASE32). Lets say you have UUID numbers stored as DECIMAL and need to show them as BASE32 or vice versa:
SELECT uuid, conv(uuid, 10, 32) uuid_b32, conv(conv(uuid, 10, 32), 32, 10)
FROM database.table;
The answer above is for number conversions between distinct base. If that is not the case, as when you have a binary file stored on a blob column, them you'll probably need to do the encoding/decoding outside MySQL. You may use MIME::Base32 or the proper module on your preferred language. Anyway, you'll need to know if the field has text or binary encoded in Base32.
Related
I was working on some binary practice problems when I noticed something interesting. How does a computer differentiate between binary values. For instance 13 in binary is 1101 and 0.8125 is also 1101 in binary. Since their binary values are the same, how does a computer know which is which. Or if I were converting it back to base 10, how would I know if the number was originally 13 or 0.8125?
The computer doesn't care about the "meaning" of the binary values till the point you instruct it to use it. When you do that, you explicitly "tell" the computer what the meaning is.
The binary value in a nylocation in memory could be anything (a number, a program instruction, a floating point number, etc)... the program has to know what type to expect at that location.
Data Type is the answer.
Computer looks for the datatype.
If binary value is 1101 and data type mentioned when declaring the variable was integer, then it will be 13. If data type mentioned was float, then it will be 0.8125. If the data type mentioned was char, then 13 would be the ascii value of the character.
I hope you understood what I explained.
I have an application written in "C" - debian distro (using libmysqlclient). My application is inserting a huge number of rows in to the database (around 30.000 rows/sec, row length ~ 150 B). Insertion takes many of CPU, because the client (my app) must convert binary stream (integers, blobs) in to ASCII representation in to a valid SQL insert statement. And also the MySQL must convert this values in to binary representation and store it to the file.
And my question. Is some possibility to call SQL insert without this conversation? I have been using a classical sprintf (implemented in libc). Now, I'm using optimized version and
without calling sprintf (too many func calling). I thought I'll use a MySQL prepared statement, but it seems a prepared statement also convert '?' variables to the ASCII.
Jan
See mysql_real_query().
22.8.7.56 mysql_real_query()
int mysql_real_query(MYSQL *mysql, const char *stmt_str, unsigned long length)
Apparently, something like the below should work
#define DATA "insert into atable (acolumn) values('zero\000quotes\"\'etc ...')"
if (mysql_real_query(db, DATA, sizeof DATA - 1)) /* error */;
I'm working with some binary waveform files from various early to mid-90's HP scopes. I am trying to do a bulk conversion (we have over 5000) of the files to CSV's and then upload them into a database. I've tried hexdump, xxd, od, strings, etc. and none of them seem to work. I did hunt down a programmers manual but it's not making a whole lot of sense.
The files have a preamble line as ascii text but then the data points are in binary and for some reason nothing I try can decode them. The preamble gives the data necessary to use the binary values and calculate the correct values. It also states that the data is in WORD format.
:WAV:PRE 2,1,32768,1,+4.000000E-08,-4.9722700001108E-06,0,+2.460630E-04,+2.500000E+00,16384;:WAV:DATA #800065536^W�^W�^W�^
I'm pretty confused.
Have a look at
http://www.naic.edu/~phil/hardware/oscilloscopes/9000A_Programmer_Reference.pdf
specifically page 1-21. After ":WAV:DATA", I think the rest of the chunk above will have 65536 8-bit data bytes (the start of which is represented above by �) . The ^W is probably a delimiter, so you would have to parse that out. Just a thought.
UPDATE: I'm new to oscilloscope data collection and am trying to figure the whole thing out from scratch. So, on further digging, it looks like the data you have provided shows this:
PREamble:
- WORD format (16-bit signed integers split into 2 8-bit bytes)
- If there is a WAV:BYT section, that would specify byte order for each pair
- RAW data
- 32768 data points
- COUNT = 1 (I'm not clear on the meaning of this)
- Next 3 should be X increment, origin, reference
- Next 3 should be Y increment, origin, reference, although the manual that I pointed you at above has many more fields than just these, so you might want to consult your specific scope manual.
DATA:
- On closer examination, I don't think the ^W is a delimiter, I think it is the first byte of the pair (0010111). The � character is apparently a standard "I don't know how to represent this character" web representation. You would need to look at that character as 8 bits also.
- 65536 byte pairs of data
I'm not finding a utility that will do this for you. I think you're going to have to write or acquire some code (Perl, C, Java, Python, VB, etc.) to get this done.
I want to Bitwise-XOR a string (actually its binary representation) with a KEY.
The result of the operation should be represented as HEX.
What I have:
'a' - the UTF-8 String to be changed.
'ACF123456' - the key in HEX.
Result seen as BIGINT:
select CONV(HEX('a'), 16, 10) ^ CONV('ACF123456', 16, 10);
Result seen as HEX:
select CONV( CONV(HEX('a'), 16, 10) ^ CONV('ACF123456', 16, 10), 10, 16);
Questions:
Is the conversion above done correctly?
What happens if the string is too long (i.e instead of 'a' we have 'a veeeeeery long string')? It seems that the conv() function has a limitation (is it the 64-bit precision from the documentation)? And besides the XOR operator ^ has also a limitation, related to the nr. of bits of the returned result. Any solutions that work for any string (a stored procedure is allowed)?
Thanks.
Your conversions look fine to me.
And as you point out, both CONV() and ^ have indeed a 64-bits precision.
2^64 = 16^16, therefore strings of more than 16 hexadecimal digits should convert to integers larger than 2^64. However, such strings will be brutally (silently) truncated from the left when attempting to convert them to integers.
The point of my solution here is to slice such strings. Obviously, the result may not be displayed as an integer, but only as a string representation.
Let #input be your "string to be changed" and #key, your "key".
Assign HEX(#input) to #hex_input. No problem here since HEX() works with strings.
Slice #hex_input into 16 hexadecimal digit long strings, starting from the right
Likewise, slice #key into 16 digit long strings.
Compute the X-OR of each 64-bit slice of #hex_input with each 64-bit slice of #key, starting from the right. Use CONV(#slice, 16, 10). If either #hex_input or #key has less slices than the other string, then X-OR the remaining slices of the other string with 0.
Convert each 64-bit number resulting from the X-OR in point 4. back into an hexadecimal string with UNHEX().
Reassemble the resulting slices. This is your result.
A three-columns TEMPORARY table could be used as an array to store slices of #hex_input, #mask and the resulting slices.
Put this all together into a stored procedure, and voilà!
You sound like you have some skills in MySQL, you should be able to translate the above into real code. But I'll be happy to help if you need further guidance.
I've been asked to process some files serialized as binary (not text/JSON unfortunately) Thrift objects, but I don't have access to the program or programmer that created the files, so I have no idea of their structure, field order, etc. Is there a way using the Thrift libraries to open a binary file and analyze it, getting a list of the field types, values, nesting, etc.?
Unfortunately it appears that Thrift's binary protocol does not do very much tagging of data at all; to decode it appears to assume you have the .thrift file in hand so you know, say, the next 4 bytes are supposed to be an integer, and aren't actually the first half of a float. So it appears you are stuck with, basically, looking at the files in a hex editor (or equivalent) and trying to deduce fields based on the exact patterns you're seeing.
There are a very few helpful bits:
Each file begins with a version, protocol identifier string, and sequence number. Maps will begin with 6 bytes that identify the key and value types (first two bytes, as integer codes) plus the number of elements as a 4 byte integer. The type codes appear to be standard (the canonical location of their definitions seems to be TProtocol.h in the Thrift sources, for instance a boolean value is specified by type code 2, UTF-8 string by type code 16, and so on). Strings are prefixed by a 4 byte integer length field, and lists are prefixed by the type (1 byte) and a 4 byte length. It looks like all integer fields are saved big-endian, and floating points are saved in IEEE format (which should make doubles relatively easy to find, at least).
The TBinaryProtocol* files in Thrift have a few more helpful details; on the plus side, there are a number of different implementations so you can read the ones implemented in the language you are most comfortable with.
Sorry, I know this probably isn't that helpful but it really does appear this is all the information the Thrift binary format provides; clearly the binary format was designed with the intent that you would always know the exact protocol spec already, and that the goal was the minimize wire space, rather than make it at all easy to decode blindly.