Cassandra: Cannot parse <col_Name> as hex bytes: MarshallException - exception

I was trying my first 'Helloworld' application in Cassandra. Whenever I try to add any data to my keyspace column family I get this error:
[default#MyKeyspace] set User['ehewitt'] ['fname']='Eben';
org.apache.cassandra.serializers.MarshalException: cannot parse 'fname' as hex bytes
This is despite the fact that I have executed
[default#MyKeyspace] assume Users keys as utf8;
So the above command does not seem to have any effect at all. How do I solve this issue?

Cassandra is assuming the columns as bytes.
Check with
help assume;
assume User keys as ascii;
assume User comparator as ascii;
assume User validator as ascii;
assume User sub_comparator as ascii;
set User['ehewitt']['fname']='Eben';
Value inserted.
Elapsed time: 216 msec(s).

I had similar problem, but the cli told me that the value is what cannot be parsed.
set game_outcome['1']['userId']='123asdasd';
cannot parse '123asdasd' as hex bytes
so I tried to use utf8 function like this :
set game_outcome['1']['userId']=utf8('123asdasd');
cannot parse '123asdasd' as hex bytes
Try
set User['ehewitt'] [utf8('fname')]='Eben'
I tried to use set some assumption like this
assume validator keys as utf8;
validator not found in current keyspace.
But as you can see it did not work as well !
I hope this answer helps.

Starting the CLI
You can start the CLI using the bin/cassandra-cli script in your Cassandra installation (bin\cassandra-cli.bat on windows). If you are evaluating a local cassandra node then be sure that it has been correctly configured and successfully started before starting the CLI.
If successful you will see output similar to this:
Welcome to cassandra CLI.
Type 'help;' or '?' for help. Type 'quit;' or 'exit;' to quit.
You must then specify a system to connect to:
connect localhost/9160;
Creating a Keyspace
We first create a keyspace to run our examples in.
create keyspace Twissandra;
Selecting the keyspace to user
We must then select our example keyspace as our new context before we can run any queries.
use Twissandra;
To Create A Column
We can then create a column to play with.
create column family User with comparator = UTF8Type;
For the later examples to work you must also update the schema using the following command. This will set the return type for the first and last name to make them human readable. It will also add and index for the age field so that you filter your gets using the Users name field.
update column family User with
column_metadata =
[
{column_name: first, validation_class: UTF8Type},
{column_name: last, validation_class: UTF8Type},
{column_name: age, validation_class: UTF8Type, index_type: KEYS}
];
To Add Data
To add data we want to into our new column we must first specify our default key type otherwise we would have to specify it for each key using the format [utf8('keyname')] this is probably advisable if you have mixed key types but makes simple cases harder to read.
So we run the command below, which will last the length of you cli session. On quitting and restarting we must run it again.
assume User keys as utf8;
and then we add our data.
set User['jsmith']['first'] = 'John';
set User['jsmith']['last'] = 'Smith';
set User['jsmith']['age'] = '38';
If you get the error like this cannot parse 'John' as hex bytes, then it likely you either haven't set your default key type or you haven't updated your schema as in the create column example.
To Update Data
If we need to update a value we simply set it again.
set User['jsmith']['first'] = 'Jack';
To Get Data
Now let's read back the jsmith row to see what it contains:
get User['jsmith'];
The get command uses API#get_slice
To Query Data
get User where age = '12';

Related

MySQL AES-CBC decryption key padding - what if String is too small? (no multiple of 16)

I successfully encrypted Strings in MySQL using AES-128-CBC algorithm. The official documentation and tutorials helped a lot. From my understanding, MySQL uses key padding PKCS#7 in order to fill the binary data in chunks of 16 Byte-size.
Problem: I am not able to decrypt Data using MySQL, although these Data is successfully decrypted using online decryption tools?!
Data:
0x568aed71e43af834900bec738e08c4fa2637b8915fb401fd6296f19c3aeeceebc3164b967cd5445e6aafe90f416314191cb1839210b7cd2efe168911fd465dab56ccda9c82862b90f29353ab57532b49
This is a telegram that I received from a smoke detector. I can easyly decrypt it via online tools, e.g. 'online-domain-tools.com':
AES Key:
0xDDCCFA1971D96B759B6A20400E00A535
iv:
2515485001000C1A2323232323232323
Result in online-AES-decryptor-webpage: 0x2f2f0bfd0f070101046d3209972702fd17000082206c9225426c90278440ff2c000f13ff8250fd61000082506c01018260fd6105008360fd31e87a0082606c9b258270fd61070082706c97272f2f2f2f
I can confirm the result beeing correct, due to the prefix (2f2f) and suffix (2f2f2f2f).
I tried the same decryption in MySQL using aes-128-CBC algorithm, but the result is NULL?!?!
SET ##SESSION.block_encryption_mode = 'aes-128-cbc';
SET #iv = 0x2515485001000C1A2323232323232323;
SET #pass = 0xDDCCFA1971D96B759B6A20400E00A535;
SET #raw = 0x568aed71e43af834900bec738e08c4fa2637b8915fb401fd6296f19c3aeeceebc3164b967cd5445e6aafe90f416314191cb1839210b7cd2efe168911fd465dab56ccda9c82862b90f29353ab57532b49;
SELECT HEX(AES_DECRYPT(#raw,#pass, #iv));
After a weekend of frustration my only understanding was, that if I encrypt the above data within the same procedure, the Result is as follows:
Encryption:
0x568aed71e43af834900bec738e08c4fa2637b8915fb401fd6296f19c3aeeceebc3164b967cd5445e6aafe90f416314191cb1839210b7cd2efe168911fd465dab56ccda9c82862b90f29353ab57532b49ac0e75a2663f9a22b1036d03e2f9b9a6
Heureka! The Result is different to the Result that I expected (addition of 16 Bytes, bold formatting). My suggestion is, that MySQL was unable to decrypt my initial data due to key padding? If i decrypt the longer data above, the only difference is key padding bytes at the end of the original data:
Decrypted:
0x2f2f0bfd0f070101046d3209972702fd17000082206c9225426c90278440ff2c000f13ff8250fd61000082506c01018260fd6105008360fd31e87a0082606c9b258270fd61070082706c97272f2f2f2f10101010101010101010
The bold bytes at the end is key padding data, from my understanding.
Other decryption result in MySQL (no key padding?!):
SET ##SESSION.block_encryption_mode = 'aes-128-cbc';
SET #iv = 0x2515485001000C1A2323232323232323;
SET #pass = 0xDDCCFA1971D96B759B6A20400E00A535;
SET #raw = 0x568aed71e43af834900bec738e08c4fa2637b8915fb401fd6296f19c3aeeceebc3164b967cd5445e6aafe90f416314191cb1839210b7cd2efe168911fd465dab56ccda9c82862b90f29353ab57532b49ac0e75a2663f9a22b1036d03e2f9b9a6;
SELECT HEX(AES_DECRYPT(#raw,#pass, #iv));
Is there a way to 'add' the missing bytes to encrypted data, in order to enable MySQL to decrypt it?
Thanks a lot!

Unable to change the value of enum field in MySQL when manually typing a query

I have an enum column in a MySQL table called site_status:
From my understanding, updating the value of such a field is exactly the same process as changing value of a varchar or int or any other field.
The only "limitation" is that as a value I obviously have to set one from predefined values - as this is the nature of enum types. Right?
So I want to change site_status from LIVE to FREE:
I type:
update db_name.db_table set site_status = 'FREE' where site_id = 501;
Pretty easy query. Unfortunately does nothing. It also doesn't produce any error. Simply:
0 rows affected.
What am I doing wrong? When I manually click via GUI in Sequel Pro, the value is changed to FREE with no issues. I see a dropdown from which I pick another option.

SAS pass through - Extract from MySQL does not work

I'm trying to build a Data Integration job uses pass through to extract data from a view in a MySQL database.
Wev'e been using pass through a lot in the project, mostly extracting data from Redshift,
however with MySQL I was not able to do make it work properly.
It keeps complaining a table is missing even though when pass through is off, view is found and data is extracted...
tried every trick I know, starting from enabling case-sensitive DBMS object names, to manually remove single/double quotes from the statement just in case MySQL confuses confuses it with something else...
No luck.
ODBC driver is [MySQL][ODBC 5.3(a) Driver][mysqld-5.5.53].
Ran on a Windows environment.
Any idea how to solve this?
Thank you in advance.
EDIT
So, first of all, one correction (even though not that important - I extract from a view, not a table).
This is the code generated by SAS Create Table transformation, pass through enabled. I only put an asterisk instead of the full list of columns:
proc sql;
connect to ODBC
(
READBUFF=10000 DATASRC="cmp.web_api" AUTHDOMAIN="MYSQL_CMP_Auth"
);
create table work."W7ZZZKOC"n as
select
*
from connection to ODBC
(
select
V_BI_ACCOUNT.ACCOUNT_NAME,
V_BI_ACCOUNT.ACQUISITION_SOURCE__C,
V_BI_ACCOUNT.ZUORA__ACTIVE__C,
V_BI_ACCOUNT.ADDRESS_LINE_1__C,
V_BI_ACCOUNT.ADDRESS_LINE_2__C,
V_BI_ACCOUNT.ADDRESS_LINE_3__C,
V_BI_ACCOUNT.AGREEMENT_DATE,
V_BI_ACCOUNT.AGREEMENT_LEGAL_CLAUSE_1__C,
V_BI_ACCOUNT.AGREEMENT_LEGAL_CLAUSE_2__C,
V_BI_ACCOUNT.PERSONBIRTHDATE,
V_BI_ACCOUNT.BLOCKED_REASON__C,
V_BI_ACCOUNT.BRAND__C,
V_BI_ACCOUNT.CPN__C,
V_BI_ACCOUNT.ACCCREATEDBYID,
V_BI_ACCOUNT.ACCCREATEDDATE,
V_BI_ACCOUNT.CURRENCY_PREFERENCE__C,
V_BI_ACCOUNT.CUSTOMER_FULL_NAME__PC,
V_BI_ACCOUNT.ACCOUNTID,
V_BI_ACCOUNT.ZUORA__CUSTOMERPRIORITY__C,
V_BI_ACCOUNT.DELIVERY_SALUTATION__C,
V_BI_ACCOUNT.DISPLAY_NAME,
V_BI_ACCOUNT.PERSONEMAIL,
V_BI_ACCOUNT.EMAILKEY__C,
V_BI_ACCOUNT.FACEBOOKKEY,
V_BI_ACCOUNT.FIRSTNAME,
V_BI_ACCOUNT.GENDER__C,
V_BI_ACCOUNT.PHONE,
V_BI_ACCOUNT.ACCLASTACTIVITYDATE,
V_BI_ACCOUNT.ACCLASTMODIFIEDDATE,
V_BI_ACCOUNT.LASTNAME,
V_BI_ACCOUNT.OTHER_EMAIL__C,
V_BI_ACCOUNT.PI_TYPE__C,
V_BI_ACCOUNT.ACCPARENTID,
V_BI_ACCOUNT.POSTCODE__C,
V_BI_ACCOUNT.PRIMARY_ACCOUNT_OF_THIS_CUSTOMER,
V_BI_ACCOUNT.ACCPRIMARY__C,
V_BI_ACCOUNT.ACCREASON_FOR_STATUS__C,
V_BI_ACCOUNT.ZUORA__SLA__C,
V_BI_ACCOUNT.ZUORA__SLASERIALNUMBER__C,
V_BI_ACCOUNT.SALUTATION,
V_BI_ACCOUNT.ACCSYSTEMMODSTAMP,
V_BI_ACCOUNT.PERSONTITLE,
V_BI_ACCOUNT.ZUORA__UPSELLOPPORTUNITY__C,
V_BI_ACCOUNT.X_CODE__C,
V_BI_ACCOUNT.ZUORA__ACCOUNT_ID__C,
V_BI_ACCOUNT.ZUORA__PAYMENTMETHODID__C,
V_BI_ACCOUNT.CITY,
V_BI_ACCOUNT.ORIGINAL_CREATED_DATE,
V_BI_ACCOUNT.SOURCE_SYSTEM_ID,
V_BI_ACCOUNT.STATUS,
V_BI_ACCOUNT.ZUORA__CONTACT_ID,
V_BI_ACCOUNT.ACCISDELETED,
V_BI_ACCOUNT.BILLING_ACCOUNT_NAME,
V_BI_ACCOUNT.ACZCREATEDDATE,
V_BI_ACCOUNT.ACZSYSTEMMODSTAMP,
V_BI_ACCOUNT.ACZLASTACTIVITYDATE,
V_BI_ACCOUNT.ZUORA__ACCOUNT__C,
V_BI_ACCOUNT.ZUORA__ACCOUNTNUMBER__C,
V_BI_ACCOUNT.ZUORA__AUTOPAY__C,
V_BI_ACCOUNT.ZUORA__BALANCE__C,
V_BI_ACCOUNT.ZUORA__CREDITCARDEXPIRATION__C,
V_BI_ACCOUNT.ZUORA__CURRENCY__C,
V_BI_ACCOUNT.ZUORA__MRR__C,
V_BI_ACCOUNT.ZUORA__PAYMENTTERM__C,
V_BI_ACCOUNT.ZUORA__PURCHASEORDERNUMBER__C,
V_BI_ACCOUNT.ZUORA__LASTINVOICEDATE__C,
V_BI_ACCOUNT.COUNTRY_NAME,
V_BI_ACCOUNT.COUNTRY_CODE,
V_BI_ACCOUNT.FAVOURITE_FOOTBALL_CLUB,
V_BI_ACCOUNT.COUNTY
from
web_api.V_BI_ACCOUNT as V_BI_ACCOUNT
);
%rcSet(&sqlrc);
disconnect from ODBC;
quit;
And again, when I extract data without pass through - works successfully,
I found out the problem was a column name exceeds 32 positions.
As SAS supports up column names up to 32,
the query fails to find PRIMARY_ACCOUNT_OF_THIS_CUSTOMER as the original column name is PRIMARY_ACCOUNT_OF_THIS_CUSTOMER__C.
EDIT
One more thing I found out is, MySQL doesn't like specifying schema name nor aliases.
Therefore,
From clause to only specify table name i.e : 'from v_bi_account' rather than 'web_api.v_bi_account'
and do not use aliases i.e use 'from v_bi_account' rather than 'from v_bi_account as v_bi_account'
Thank you guys so much for your help.

MySQL REPLACE in UPDATE does not work properly

The following query:
select replace(`Abilities`, 'export_import', 'auto') from fl_account_types;
gives me 'auto,listings' correct replacement from Abilities column. However, when I execute:
update fl_account_types set `Abilities` = replace(`Abilities`, 'export_import', 'autos');
MySQL just omits 'export_import' string and replaces Abilities with 'listings' string.
What could be the reason?
The problem was that Abilities was of type SET and I was trying to replace with a value which was not listed in a definition of it. But I still do not understand why select replace works well and why MySQL do not throw an error.

Unknown character set index for field received from server

I have an instance of MySQL 5.0.4.1 with an application written in Hibernate. On one of the pages, I get the following error message in the server log:
Unknown character set index for field '123' received from server.at com.mysql.jdbc.Connection.getCharsetNameForIndex(Connection.java:1664)at com.mysql.jdbc.Field.(Field.java:144)at com.mysql.jdbc.MysqlIO.unpackField(MysqlIO.java:506)at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:280)at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1319)at com.mysql.jdbc.MysqlIO.sqlQuery(MysqlIO.java:1218)at com.mysql.jdbc.Connection.execSQL(Connection.java:2233)at com.mysql.jdbc.Connection.execSQL(Connection.java:2193)at com.mysql.jdbc.Connection.execSQL(Connection.java:2174)at com.mysql.jdbc.Connection.setAutoCommit(Connection.java:536)at org.apache.commons.dbcp.DelegatingConnection.setAutoCommit(DelegatingConnection.java:268)at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.setAutoCommit(PoolingDataSource.java:293)at org.hibernate.transaction.JDBCTransaction.toggleAutoCommit(JDBCTransaction.java:194)at org.hibernate.transaction.JDBCTransaction.rollbackAndResetAutoCommit(JDBCTransaction.java:186)at org.hibernate.transaction.JDBCTransaction.rollback(JDBCTransaction.java:162)at org.springframework.orm.hibernate3.HibernateTransactionManager.doRollback(HibernateTransactionManager.java:603)at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:579)at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:556)at org.springframework.transaction.interceptor.TransactionAspectSupport.doCloseTransactionAfterThrowing(TransactionAspectSupport.java:284)at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:100)at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:170)at org.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:66)at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:170)at org.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:66)at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:170)at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:176)at $Proxy6.getDataFromDatabase(Unknown Source)at org.myCompany.myAction.load(Unknown Source)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)at java.lang.reflect.Method.invoke(Method.java:324)at org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:274)at org.apache.struts.actions.DispatchAction.execute(DispatchAction.java:194)at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:419)at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:224)at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:414)at javax.servlet.http.HttpServlet.service(HttpServlet.java:787)at javax.servlet.http.HttpServlet.service(HttpServlet.java:908)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193)at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:264)at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107)at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72)at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:110)at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:217)at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:229)at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:148)at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:98)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:213)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193)at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:280)at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:509)at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:218)at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:509)at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:209)at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:509)at com.iplanet.ias.web.connector.nsapi.NSAPIProcessor.process(NSAPIProcessor.java:157)at com.iplanet.ias.web.WebContainer.service(WebContainer.java:579)
This error occurs on a production environment so I don't know which query produced the problem, but I do know that none of my tables has a column named '123'. Do you have any suggestions about what might be causing this problem?
Edit: As a followup, I found that this behavior is a known bug in MySQL 5.0.4, but I haven't found a good wokaround since I don't have the option of upgrading mySQL.
Could be that somehow the character set for the field has been set to an invalid value, run this SQL on the server with the problem replacing the table name and the column 'Collation' will show the characterset for each varchar/char fields
SHOW FULL COLUMNS IN table_name;
You can then change the character set of a field by using the following:
ALTER TABLE t MODIFY col1 VARCHAR(50) CHARACTER SET latin1;
Internally, MySQL is mapping out the indexes across the columns. Normally this is done using integers that represent the column index. Most indexes are actually a compound index once built (col 1 + col 3) which form something like field 13.
Likely this happened when migrating data from dev into production when the stack is not an exact replica.
As 3urdoch mentioned you can pull the charset using internal MySQL function; and then change the table's charset to a compatible encoding.
Alternatively (where I'm willing to bet this issue came from) you can check if the loaded driver for the Production matches the loaded driver for the Other server. This will prevent re-occurring issue if the Development/Other server is still being used for testing, and migrated into Production.