I am trying to insert the contents of an JSON to a MySql database using Mule ESB. The JSON looks like:
{
"id":106636,
"client_id":9999,
"comments":"Credit",
"salesman_name":"Salvador Dali",
"cart_items":[
{"citem_id":1066819,"quantity":3},
{"citem_id":1066820,"quantity":10}
]
}
On mule I want to insert all data on a step like:
Insert INTO order_header(id,client_id,comments,salesman_name)
Insert INTO order_detail(id,citem_id,quantity)
Insert INTO order_detail(id,citem_id,quantity)
Currently i have come this far on Mule:
MuleSoft Flow
Use Bulk Execute operation of Database Connector.
You will insert into multiple tables.
for ex :
Query text
Insert INTO order_header(payload.id,payload.client_id,payload.comments,payload.salesman_name);
Insert INTO order_detail(payload.id,payload.cart_items[0].citem_id,payload.cart_items[0].quantity); etc..
There is an excellant article here http://www.dotnetfunda.com/articles/show/2078/parse-json-keys-to-insert-records-into-postgresql-database-using-mule
that should be of help. You may need to modify as you need to write the order_header data first and then use a collection splitter for the order_detail and wrap the whole in a transaction.
Ok. Since, you have already converted JSON into Object in the flow, you can refer individual values with their object reference like obj.id, obj.client_id etc.
Get a database connector next.
Configure your MySQL database in "Connector Configuration".
Operation: Choose "Bulk execute"
In "Query text" : Write multiple INSERT queries and pass appropriate values from Object (converted from JSON). Remember to separate multiple queries with semicolon (;) in Query text.
That's it !! Let me know if you face any issue. Hope it works for you..
Related
I am using syslog-ng to parse some logs that I am receiving via a csv-parser. However, I want to achieve insert operations that are a bit more complex than the conventional insert using the "destination" option in syslog-ng. Currently, my destination into MYSQL from my syslog-ng conf file looks like this:
destination d_sql_test
{
sql(
type(mysql)
host('<host>')
username('<user>')
password('<pass>')
database('<db_name>')
table('test')
columns('col1')
values('${val1}')
);
};
However, this simply just inserts the contents of val1 into the column col1. I want to be able to specify my insert "logic" as shown in the example in this question.
I am unsure as to where to actually do this, and if it is even supported by syslog-ng
I think you can do this if you can somehow make the decision within syslog-ng.
You could try to use an in-list() filter to check if the username is already listed in a file. If it is not then, you can send the log into the mysql destination, and also to another destination (possibly a program() destination) that updates the file containing the list of users, and reloads the syslog-ng to update the inlist filter.
You can write a syslog-ng template-function in Python that implements the logic somehow, and for example sets a macro to 1 in the message if it should be sent to the database. Then you can use a filter for this macro in your log path with the mysql destination.
Or if you can write a separate destination that does the work in Python: Writing syslog-ng destinations in Python
Also, you might want to post this question on the syslog-ng mailing list, where the developers notice it more easily.
My original problem is that I need to insert a lot of records to DB, so to speed up, I want to use mysqlimport which takes a file of row values and load them to specified table. So suppose I have model Book, I couldn't simply use book.attributes.values as one of the fields is a hash that is serialized to db (using serialize), so I need to know what is the format this hash will be stored in in the db. Same for time and dates fields. Any help?
How about using SQL insert statements instead of serialization?
book = Book.new(:title => 'Much Ado About Nothing', author: 'William Shakespeare')
sql = book.class.arel_table.create_insert
.tap { |im| im.insert(record.send(
:arel_attributes_with_values_for_create,
record.attribute_names)) }
.to_sql
I am trying to insert data into mysql table through mysql C client, through the step written below.
The command is of the Form : (A variable string generated at run time)
INSERT INTO department values('Statistics','Taylor',395051.74)
which is correct for MySQL.
if (mysql_query(con, command))
{
printf("Done\n");
}
printf("\n%s\n",command);
But my database shows no change. No rows get inserted, is there any way the above steps could not work?
Note that mysql_query returns a zero if it is successful, and an error code if it's unsucessful MySQL Docs. I think you might be treating it backward. So I think it's issuing an error you're not catching.
As a guess of what might be wrong, try telling it what columns you're inserting into:
INSERT INTO department (`column1`,`column2`,`column3`)
values ('Statistics','Taylor',395051.74)
Can I generate an sql insert statement from an array returned by find methods?
Thanks.
Yes, You can.
After performing the find() operation iterate your data using foreach loop and make a string based insert query with ";" as separator after each insert query. Create a file with extension as .sql and use force download component to download it.
I have some sort of code that can help you. I'll try to give you the code soon.
I'll copy past a part from the mule website guide:
<jdbc:query key="outboundInsertStatement"
value="INSERT INTO TEST (ID, TYPE, DATA, ACK) VALUES (#[map-payload:ID],
#[map-payload:TYPE],#[map-payload:DATA], #[map-payload:ACK])"/>
I am trying to do something very close to this only I want to use a custom object and not the java.util.map which i understand is what is expected.
Could I get an explanation as to what does #[map-payload:ACK] exactly means? I dont understand the syntax.
Is map-payload some sort of default type?
Could I use that syntax to use a custom object I created? (Some MesssageObj class with some fields)
The syntax:
#[evaluator:expression]
is used by the Mule Expression Evaluation framework.
If you look in the table that lists all evaluators, you'll find map-payload among may other evaluators.
So the example you have above means that:
it is expected the in-flight message will have a payload of type java.util.Map,
the values for the ID, TYPE, DATA and ACK columns in the insert query will be extracted from the map payload under eponymous keys.
Of course, feel free to use any other evaluator that better match your in-flight message payload.