I have a poll enrich which enriches a POJO with the result of an SQL query (from a MySQL database). It currently gets the brand from the POJO and then gets the name from the order matching the brand. I had to add quotes around the ${body.getBrand}, else the query would look for a column with the brand name instead of using the value. Currently it looks like this:
<pollEnrich id="_enrich1" strategyRef="merge" timeout="5000">
<simple>sql:SELECT name FROM orders WHERE brand= '${body.getBrand}'</simple>
</pollEnrich>
I want to change it because I'll probably need to create more sql queries and the current version does not work if the value contains quotes and thus is vulnerable to sql injection.
I thought prepared statements would do the trick and wanted to use a named parameter but I do not seem to be able to set the value of the parameter.
I have tried many different things like for example setting a header and change the query to have a named parameter:
<setHeader headerName="brand" id="brand">
<simple>${body.getBrand}</simple>
</setHeader>
<pollEnrich id="_enrich1" strategyRef="merge" timeout="5000">
<simple>sql:SELECT name FROM orders WHERE brand= :#brand</simple>
</pollEnrich>
but I keep getting
PreparedStatementCallback; bad SQL grammar [SELECT name FROM orders WHERE brand= ?]; nested exception is java.sql.SQLException: No value specified for parameter 1
I have also tried setting the useMessageBodyForSql option to true (since this seemed like something that might help?) but nothing I have tried seemed to work.
I have seen a lot of examples/solutions for people setting the routes with java, but I assume there must also be a solution for the blueprint xml?
If anyone got any suggestion or example that would be great.
In Camel version < 2.16, pollEnrich doesn't have access to the original exchange and therefore cannot read your header, hence the exception. This is documented here: http://camel.apache.org/content-enricher.html
Guessing from your example, a normal enrich should work too and it has access to the original exchange. Try changing 'pollEnrich' to 'enrich'.
Related
as the title suggests I am having a problem trying to use JPA Criteria with Spring Boot for a specific case.
Generally everything works but trying to search for stored data with a String property having newLine character embedded ( \n ) doesn't seem to work.
I am able to Get the data, edit them, save them through my front end, create new with multiple lines etc. but trying to search for them when for example a column is equals with 'hello\nworld' it wont work even though running this query in MySQL Workbench works returning the desired data :
select * from kerkinidb.ct_thhlastika_press_threats where description_en = 'hello\nworld';
To clarify, the way I do the search is by waiting in a Get request an argument called search which has all the properties that the user filtered. I am matching it with a Regex (which also has inside the Java 8.0 new Regex \\\\R for matching with multilines (and it works) ) then I am giving the the Service layer the Search Criteria that I matched which then passes to the Jpa Criteria Repository to parse them and generating the Predicates (matching again with Regex and \\\\R to create a final Predicate with OR and ANDs for the filtering) then triggering the query, then making another query called count to implement Pagination and finally mapping to a custom object and returning it.
I debugged every step and the final Predicate does generate the query I want, but the db doesn't return the expected data. So I am really confused since as I said the query does work in MySQL Workbench.
This is an example of the logging (I turned Spring Boot logging for MySQL logs on) generated when the Request is being triggered (in this case the stored data I have in my Table ct_thhlastika_press_threats in column description_en is a\ns\ndd so I am searching for this one as you can see instead of the example I said earlier hello\nworld :
2019-02-12 16:01:01.929 DEBUG 18368 --- [nio-8080-exec-2] org.hibernate.SQL : select ctthhlasti0_.id as col_0_0_, ctthhlasti0_.act_code as col_1_0_, ctthhlasti0_.description_en as col_2_0_, ctthhlasti0_.remarks as col_3_0_ from ct_thhlastika_press_threats ctthhlasti0_ where 1=1 and ctthhlasti0_.description_en=? order by ctthhlasti0_.id asc limit ?
2019-02-12 16:01:01.933 TRACE 18368 --- [nio-8080-exec-2] o.h.type.descriptor.sql.BasicBinder : binding parameter [1] as [VARCHAR] - [a\ns\ndd]
2019-02-12 16:01:01.944 DEBUG 18368 --- [nio-8080-exec-2] org.hibernate.SQL : select count(ctthhlasti0_.id) as col_0_0_ from ct_thhlastika_press_threats ctthhlasti0_ where 1=1 and ctthhlasti0_.description_en=?
2019-02-12 16:01:01.944 TRACE 18368 --- [nio-8080-exec-2] o.h.type.descriptor.sql.BasicBinder : binding parameter [1] as [VARCHAR] - [a\ns\ndd]
2019-02-12 16:01:01.946 TRACE 18368 --- [nio-8080-exec-2] o.h.type.descriptor.sql.BasicExtractor : extracted value ([col_0_0_] : [BIGINT]) - [0]
For anyone interested to look farther into the code you could find it Github repo. The project is for my University thesis. I made comments for adding to the Regexes (both of them) for ctThhlastikaPressThreats Controller and Repository. The db (the standard as well as the secondary for testing) need to be generated (can be generated by changing the auto-dll in application.properties).
Update
I tried using System.lineSeparator() (as suggested by #Hermann Steidel in the answer bellow) replacing the text newLine \n. But even though in the logs we can see now clearly that the value for the equals Predicate has line separations it still doesn't return the data from the DB.
Farther Explanation of my implementation
A correct Get request for the dynamic searching is like this :
http://localhost:8080/v1/ctThhlastikaPressThreats/search?search=descriptionEn~hello\nworld;#&size=10&page=0&sort=Asc
As you can see I am using a path variable called search (which has all the properties filtering requested from the user) along with 3 more for the size, page and sort.
For the search variable I am using 3 distinct characters to implement OR and AND Predicates for the final query. Those are the ~ which is to know that the property before it needs to use an equal predicate, the ; which is to be able to have multiple values for each property requested by the user and finally the # which triggers the end of this property's filtering.
Those 3 characters are being Regexed in two places. In the Controller and in the SearchRepository of (for example since we specifically started with this one -> the CtThhlastasikaPressThreats)
Finally in the SearchRepository you can see that I am triggering 2 queries, one is to get the filtered data from the db and another is to get the count of the data for Pagination purposes while also in the end mapping to a custom DTO.
Steps to reproduce :
After generating the db, change for the 2 Regexes of CtThhlastikaPressThreats. For the Controller to be (\w+?)(~|<|>)([(!-/.\\\\R 0-9\p{L});]+)?# and for the SearchRepository to be ([(!-/.\\\\R 0-9\p{L})]+).
Then you can use the example of the request I have above when having in the db saved for the specific table and for column descriptionEn with value of hello\nworld for example or whatever value you put (also change it into the request).
Final thing I tried but wasn't the solution :
Put in the CtThhlastikaPressThreatsSearchRepository in the method search (it is after line 61) above the :
predicate = builder.equal(root.get(param.getKey()), match.toString());
Make it as :
match = match.toString().replace("\\n", System.lineSeparator());
predicate = builder.equal(root.get(param.getKey()), match.toString());
This will basically change the value from being hellow\nworld to become hello\r\nworld so I guess it still isn't the desired solution. Thinking that in the db it is stored as \n for the lineSeparators.
Now in the logs you can see that when you trigger the Get Request again the VARCHAR value of descriptionEn is indeed now with line separations instead of the \n (which still should be recognized by MySQL) text.
Final thoughts
I believe since even this
select * from kerkinidb.ct_thhlastika_press_threats where description_en = 'hello\nworld';
works in MySQL Workbench, that something in between might be ruining the request when trying to also include newLine char or lineSeparators.
If there is any idea of why it doesn't work as intended please share to try it out.
Thank you for your time
I got it to work. Probably not the answer you wanted but, if you replace your incoming "\n"s with the system line separator, it will get you want you want.
search = search.replace("\\n", System.getProperty("line.separator"));
try {
return ctThhlastikaPressThreatsService.searchCtThhlastikaPressThreats(producedSearchCriterias(search), size, page, sort);
If you look at the params value logging, you'll see the param is now
extracted value ([col_2_0_] : [VARCHAR]) - [hello
world]
instead of
extracted value ([col_2_0_] : [VARCHAR]) - [hello\nworld]
It's like I suspected, somewhere in search pipeline, it's escaping the "\n". I'm not suggesting you do what I suggest right there on the controller, I just wanted to show you that this is what needs to happen (somewhere) for it to work.
I'm trying to build a Data Integration job uses pass through to extract data from a view in a MySQL database.
Wev'e been using pass through a lot in the project, mostly extracting data from Redshift,
however with MySQL I was not able to do make it work properly.
It keeps complaining a table is missing even though when pass through is off, view is found and data is extracted...
tried every trick I know, starting from enabling case-sensitive DBMS object names, to manually remove single/double quotes from the statement just in case MySQL confuses confuses it with something else...
No luck.
ODBC driver is [MySQL][ODBC 5.3(a) Driver][mysqld-5.5.53].
Ran on a Windows environment.
Any idea how to solve this?
Thank you in advance.
EDIT
So, first of all, one correction (even though not that important - I extract from a view, not a table).
This is the code generated by SAS Create Table transformation, pass through enabled. I only put an asterisk instead of the full list of columns:
proc sql;
connect to ODBC
(
READBUFF=10000 DATASRC="cmp.web_api" AUTHDOMAIN="MYSQL_CMP_Auth"
);
create table work."W7ZZZKOC"n as
select
*
from connection to ODBC
(
select
V_BI_ACCOUNT.ACCOUNT_NAME,
V_BI_ACCOUNT.ACQUISITION_SOURCE__C,
V_BI_ACCOUNT.ZUORA__ACTIVE__C,
V_BI_ACCOUNT.ADDRESS_LINE_1__C,
V_BI_ACCOUNT.ADDRESS_LINE_2__C,
V_BI_ACCOUNT.ADDRESS_LINE_3__C,
V_BI_ACCOUNT.AGREEMENT_DATE,
V_BI_ACCOUNT.AGREEMENT_LEGAL_CLAUSE_1__C,
V_BI_ACCOUNT.AGREEMENT_LEGAL_CLAUSE_2__C,
V_BI_ACCOUNT.PERSONBIRTHDATE,
V_BI_ACCOUNT.BLOCKED_REASON__C,
V_BI_ACCOUNT.BRAND__C,
V_BI_ACCOUNT.CPN__C,
V_BI_ACCOUNT.ACCCREATEDBYID,
V_BI_ACCOUNT.ACCCREATEDDATE,
V_BI_ACCOUNT.CURRENCY_PREFERENCE__C,
V_BI_ACCOUNT.CUSTOMER_FULL_NAME__PC,
V_BI_ACCOUNT.ACCOUNTID,
V_BI_ACCOUNT.ZUORA__CUSTOMERPRIORITY__C,
V_BI_ACCOUNT.DELIVERY_SALUTATION__C,
V_BI_ACCOUNT.DISPLAY_NAME,
V_BI_ACCOUNT.PERSONEMAIL,
V_BI_ACCOUNT.EMAILKEY__C,
V_BI_ACCOUNT.FACEBOOKKEY,
V_BI_ACCOUNT.FIRSTNAME,
V_BI_ACCOUNT.GENDER__C,
V_BI_ACCOUNT.PHONE,
V_BI_ACCOUNT.ACCLASTACTIVITYDATE,
V_BI_ACCOUNT.ACCLASTMODIFIEDDATE,
V_BI_ACCOUNT.LASTNAME,
V_BI_ACCOUNT.OTHER_EMAIL__C,
V_BI_ACCOUNT.PI_TYPE__C,
V_BI_ACCOUNT.ACCPARENTID,
V_BI_ACCOUNT.POSTCODE__C,
V_BI_ACCOUNT.PRIMARY_ACCOUNT_OF_THIS_CUSTOMER,
V_BI_ACCOUNT.ACCPRIMARY__C,
V_BI_ACCOUNT.ACCREASON_FOR_STATUS__C,
V_BI_ACCOUNT.ZUORA__SLA__C,
V_BI_ACCOUNT.ZUORA__SLASERIALNUMBER__C,
V_BI_ACCOUNT.SALUTATION,
V_BI_ACCOUNT.ACCSYSTEMMODSTAMP,
V_BI_ACCOUNT.PERSONTITLE,
V_BI_ACCOUNT.ZUORA__UPSELLOPPORTUNITY__C,
V_BI_ACCOUNT.X_CODE__C,
V_BI_ACCOUNT.ZUORA__ACCOUNT_ID__C,
V_BI_ACCOUNT.ZUORA__PAYMENTMETHODID__C,
V_BI_ACCOUNT.CITY,
V_BI_ACCOUNT.ORIGINAL_CREATED_DATE,
V_BI_ACCOUNT.SOURCE_SYSTEM_ID,
V_BI_ACCOUNT.STATUS,
V_BI_ACCOUNT.ZUORA__CONTACT_ID,
V_BI_ACCOUNT.ACCISDELETED,
V_BI_ACCOUNT.BILLING_ACCOUNT_NAME,
V_BI_ACCOUNT.ACZCREATEDDATE,
V_BI_ACCOUNT.ACZSYSTEMMODSTAMP,
V_BI_ACCOUNT.ACZLASTACTIVITYDATE,
V_BI_ACCOUNT.ZUORA__ACCOUNT__C,
V_BI_ACCOUNT.ZUORA__ACCOUNTNUMBER__C,
V_BI_ACCOUNT.ZUORA__AUTOPAY__C,
V_BI_ACCOUNT.ZUORA__BALANCE__C,
V_BI_ACCOUNT.ZUORA__CREDITCARDEXPIRATION__C,
V_BI_ACCOUNT.ZUORA__CURRENCY__C,
V_BI_ACCOUNT.ZUORA__MRR__C,
V_BI_ACCOUNT.ZUORA__PAYMENTTERM__C,
V_BI_ACCOUNT.ZUORA__PURCHASEORDERNUMBER__C,
V_BI_ACCOUNT.ZUORA__LASTINVOICEDATE__C,
V_BI_ACCOUNT.COUNTRY_NAME,
V_BI_ACCOUNT.COUNTRY_CODE,
V_BI_ACCOUNT.FAVOURITE_FOOTBALL_CLUB,
V_BI_ACCOUNT.COUNTY
from
web_api.V_BI_ACCOUNT as V_BI_ACCOUNT
);
%rcSet(&sqlrc);
disconnect from ODBC;
quit;
And again, when I extract data without pass through - works successfully,
I found out the problem was a column name exceeds 32 positions.
As SAS supports up column names up to 32,
the query fails to find PRIMARY_ACCOUNT_OF_THIS_CUSTOMER as the original column name is PRIMARY_ACCOUNT_OF_THIS_CUSTOMER__C.
EDIT
One more thing I found out is, MySQL doesn't like specifying schema name nor aliases.
Therefore,
From clause to only specify table name i.e : 'from v_bi_account' rather than 'web_api.v_bi_account'
and do not use aliases i.e use 'from v_bi_account' rather than 'from v_bi_account as v_bi_account'
Thank you guys so much for your help.
So, I am trying to execute a query using ArcGIS API, but it should match any Json queries. I am kind of new to this query format, so I am pretty sure I must be missing something, but I can't figure out what it is.
This page allows for testing queries on the database before I actually implement them in my code. Features in this database have several fields, including OBJECTID and Identificatie. I would like to, for example, select the feature where Identificatie = 1. If I enter this in the Where field though (Identificatie = 1) an error Failed to execute appears. This happens for every field, except for OBJECTID. Querying where OBJECTID = 1 returns the correct results. I am obviously doing something wrong, but I don't get it why OBJECTID does work here. A brief explanation (or a link to a page documenting queries for JSON, which I haven't found), would be appreciated!
Identificatie, along with most other fields in the service you're using, is a string field. Therefore, you need to use single quotes in your WHERE clause:
Identificatie = '1'
Or to get one that actually exists:
Identificatie = '1714100000729432'
OBJECTID = 1 works without quotes because it's a numeric field.
Here's a link to the correct query. And here's a link to the query with all output fields included.
I am working on a problem where I need to add an OR clause to a set of existing conditions. The current conditions are built in a hash in a method and at the end, they are used in the where clause. Here is a simplified example:
...
conds.merge!({:users => {:archived => false}})
Model.where(conds)
I am trying to add an OR clause to the current set of conditions so it would be something like '(conditions) OR new_condition'. I'd like to add the OR statement without converting each addition to the conds hash into a string. That would be my last option. I was hoping someone has done something like this before (without using Arel). I seem to recall in Rails 2 there was a way to parse a conditions hash using a method from the model (something like Model.some_method(conds) would produce the where clause string. Maybe that would be a good option to just add the OR clause on to that string. Any ideas are appreciated. Thank you for your help!
I found a way to do what I needed. Instead of changing all of the conditions that I am building, I am parsing the conditions to SQL using sanitize_sql_for_conditions. This is a private method in ActiveRecord, so I had to put a method on the model to allow me to access it. Here is my model method:
def self.convert_conditions_hash_to_sql(conditions)
self.sanitize_sql_for_conditions(conditions)
end
So, once I convert my conditions to text, I can add my OR clause (along with the appropriate parentheses) to the end of the original conditions. So, it would go something like this:
Model.where('(?) OR (model.type = ? AND model.id IN(?))', Model.convert_conditions_hash_to_sql(conds), model_type, model_id_array)
In my application am using JPA entity manager to persist data/fetch data.
em.executeQuery("select * from file_calender_mapping where start_date between :start and :end");
em.setParameter("start",startDate)//startDate is an date object
em.setParameter("end",endDate)//endDate is an date object
List fmlist=em.execute();
The proble is just like this,
"select * from file_calender_mapping where start_date between start and end"
when am passing some date as start= "2011-08-03 05:08:00",and end="2011-08-04 06:08:00"
then the mysql return one row having the start time ="2011-08-03 05:30:00",its good,But
when my application executing such query it dose not returning any row.Actually what i have seen that my application returning value for two different date,but not for same date different time,thats the main problem.
One another thing is my "start" field for Table "file_calender_mapping" datatype is "timestamp".
So what i was thinking that ther may be some problem on JPA/Hibernate
You can try to specify the exact types of parameters as follows:
em.setParameter("start", startDate, TemporalType.TIMESTAMP);
em.setParameter("end",endDate, TemporalType.TIMESTAMP);
I have the strong feeling that you're confusing EntityManager.createQuery() with EntityManager.createNativeQuery() and you're somehow capturing all the exceptions, which somehow makes you don't receive anything back.
I'm assuming that, because I don't think you have a class named file_calender_mapping.
edit
The documentation will explain it better than I do, but a JPA QL query is transformed to the navite sql of the DB using the mapping, while a native query is send as it's to the DB.
Again, I suggest you to read the documentation, it's quite useful.