Normally we check if replication on slaves is working via the console command
SHOW SLAVE STATUS \G;
I would like to incorporate this functionality into a servlet reporting application.
However, hibernate does not seem to allow this :
createSQLQuery("SHOW SLAVE STATUS");
...executing query...
java.lang.UnsupportedOperationException: not yet implemented for SQL queries
I'm sure it's possible to achieve this using native JDBC, however maybe there is a better way?
Environment: Linux 2.6.18, Tomcat 6/7, Hibernate 3.4, Java 1.6, MySQL 5
Note: I'm not interested in solutions where we insert timestamps on master.
When you use createSQLQuery to run a native SQL query, you have to tell Hibernate how to convert the results into Java objects. The easiest way in your case is to add
query.setResultTransformer(Criteria.ALIAS_TO_ENTITY_MAP);
to the query before executing it. (If the query returned already-mapped Entities, you could just use addEntity().)
I haven't tried hard enough to make this work with Hibernate, but if that fails, you can still use a good old-fashioned JDBC connection:
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
public class Test {
static final String DB_ADDRESS = "localhost";
static final String DB_NAME = "mysql";
static final String DB_USER = "root";
public static void main (String[] args) {
// get password
String password = "";
if (args!=null && args.length>0) {
password = args[0];
}
try {
Class.forName("com.mysql.jdbc.Driver").newInstance();
} catch (Exception ex) {
// handle the error
System.out.println("Driver issue: " + ex.getMessage());
System.out.println("Driver issue: " + ex.getClass().toString());
}
// connect to database
try {
Connection conn =
DriverManager.getConnection("jdbc:mysql://"+DB_ADDRESS+"/"+DB_NAME+"?autoReconnect=true",DB_USER,password);
// Do something with the Connection
System.out.println("Connection: " + conn);
Statement stmt = conn.createStatement();
ResultSet RS = stmt.executeQuery("SHOW TABLES");
while (RS.next()) {
System.out.println("table: '" + RS.getString(1) + "'");
}
// disconnect from database
conn.close();
stmt.close();
} catch (SQLException ex) {
// handle any errors
System.out.println("SQLException: " + ex.getMessage());
System.out.println("SQLState: " + ex.getSQLState());
System.out.println("VendorError: " + ex.getErrorCode());
}
}
}
This would return:
Connection: com.mysql.jdbc.JDBC4Connection#528acf6e
table: 'columns_priv'
table: 'db'
table: 'event'
table: 'func'
table: 'general_log'
table: 'help_category'
table: 'help_keyword'
table: 'help_relation'
table: 'help_topic'
table: 'host'
table: 'ndb_binlog_index'
table: 'plugin'
table: 'proc'
table: 'procs_priv'
table: 'proxies_priv'
table: 'servers'
table: 'slow_log'
table: 'tables_priv'
table: 'time_zone'
table: 'time_zone_leap_second'
table: 'time_zone_name'
table: 'time_zone_transition'
table: 'time_zone_transition_type'
table: 'user'
Needless to say, the key thing is to ensure your user has sufficient privileges.
This works for me..
SQLQuery query =session.createSQLQuery("show slave status");
ArrayList<Object[]> results = (ArrayList<Object[]>)query.list();
//there is only one row returned
Object [] result = results.get(0);
//Seconds Behind Master should always be the last value in the row.
String secondsBehindMaster = ""+result[result.length-1];
Ensure DB user has REPLICATION CLIENT permission to make this query.
GRANT REPLICATION CLIENT ON *.* TO 'db_user'#'db_server' IDENTIFIED BY 'db_password';
As the message of the UnsupportedOperationException exception suggests, getReturnTypes() is not implemented by the SQLQuery object returned by createSQLQuery().
By the way, the phrasing of your question is misleading, you do not receive the exception on calling createSQLQuery(), but a few lines later.
When you issue native SQL, you will retrieve rows as a list of generic Object[] (unless you explicitely provide Hibernate with a mapping). Typically, the types will be known at compile-time.
See here for more information.
Related
I am using Spring Boot to build a scheduled-job data processing application. The main logic would be in a scheduled job that takes a batch of records and process them. I should be running 2 instances of the application that should not pick the same record twice. I tried to utilize the PESSIMISTIC LOCK with NO WAIT to resolve any records selection conflict.
Things are not working as expected. Both instances are picking the same records, although I was expecting only one instance to lock and process a few records and the other instance skip what was locked by the first instance.
Spring Boot version: 2.2.4.RELEASE
Database: MySQl
First I tried using the #Lock and #QueryHint annotations:
#Lock(value = LockModeType.PESSIMISTIC_WRITE) // adds 'FOR UPDATE' statement
#QueryHints(value={#QueryHint(name = "javax.persistence.lock.timeout", value = LockOptions.SKIP_LOCKED+"")})
Page<Transaction> findByStatus(String status, Pageable pageable);
Even with WAIT_FOREVER, there is no change in behavior as if #QueryHints are totally ignored..
The other option I tried is using NativeQuery:
#Query(value ="select * from transaction t where t.status = ?1 limit ?2 for update SKIP LOCKED",
countQuery="select count(*) from transaction t where t.status = ?1",
nativeQuery = true)
List<Transaction> findByStatusNQ(String status, Integer pageSize);
Same behavior. No locking, both app instances are selecting the same set of data
This is the defined entity:
#Entity
public class Transaction {
#Id
private Long id;
private String description;
private String status;
private String managedBy;
#Temporal(TemporalType.TIMESTAMP)
private Date manageDate;
...
}
The caller service component is annotated with #Transactional to enforce creating new transaction for each execution:
#Transactional(propagation = Propagation.REQUIRES_NEW)
public List<Transaction> updateTrxStatus(String oldStatus,String newStatus){
List<Transaction> trxs = this.executeUsingNQ(oldStatus);
if(trxs.size()>0) {
logger.info( "Start updating Data");
trxs.forEach(transaction -> {
transaction.setStatus(newStatus);
transaction.setManagedBy(instanceName);
transaction.setManageDate(new Date(System.currentTimeMillis()));
});
}else{
logger.info(" Nothing to process");
}
return trxs;
}
#Transactional(propagation = Propagation.REQUIRED)
public List<Transaction> executeUsingNQ(String oldStatus){
List<Transaction> trxs = trxRepo.findByStatusNQ(oldStatus,2);
return trxs;
}
#Transactional(propagation = Propagation.REQUIRED)
public List<Transaction> executeWithPage(String oldStatus){
Pageable firstPageWithTwoElements = PageRequest.of(0, 2);
Page<Transaction> trxs = trxRepo.findByStatus(oldStatus, firstPageWithTwoElements);
return trxs.getContent();
}
Hopefully someone can help identifying whether there is some coding issue or missing coniguration!!!!
It runs that the issue was caused by using an incorrect Dialect with MySql. That version of Dialect "MySQLDialect" assumes "MyISAMStorageEngine" as a default storage engine while creating tables. That engine does not support any type of transactions.
The only storage engine that supports transactions is "InnoDB" which is being selected as the default choice when using other Dialects like "MySQL55Dialect", "MySQL57Dialect" or "MySQL8Dialect"
I'm using sails.js and sails-MySQL and I want to connect to a MySQL database. I have tables, functions, and procedures in my workbench. now I want to know that is it necessary to create model files and define my schema again?
Yes, you can connect to your DB without defining models. However bare in mind that you will have to write raw queries every time. So first you need to define your DB connection in your datastores.js file. Then you can do the following in some of your controllers when you want to get something from your DB (say you have a table users and you want to get all of them):
var myDBStore = sails.getDatastore(); //gets the default datastore.
var query = "SELECT * FROM users;";
myDBStore.sendNativeQuery(query).exec(function (err, nativeResult) {
if (err) {
return res.send(err);
}
return res.send(nativeResult.rows);
});
or using the modern way in an async function:
var myDBStore = sails.getDatastore(); //gets the default datastore.
var query = "SELECT * FROM users;";
var nativeResult;
try {
nativeResult = await myDBStore.sendNativeQuery(query);
} catch (err) {
return res.send(err);
}
return res.send(nativeResult.rows);
More info here: https://sailsjs.com/documentation/reference/waterline-orm/datastores in section "Using datastores without a model"
right now, I am kinda frustrated and I hope someone can help me and point me into the right direction.
I have an "old" project which uses the mysql statements for connection to database, etc.
Within this project I have the following:
An index file containing
*
* load configuration and connect to database
*/
$projectConfiguration = new projectConfiguration();
$dbconnect = $projectConfiguration->connect($projectConfiguration->databaseHost, $projectConfiguration->databaseName, $projectConfiguration->databaseUser, $projectConfiguration->databasePass);
// load controller
$ReqMod = FatFramework\Functions::getRequestParameter("mod");
if (!$ReqMod) {
$ReqMod = FatFramework\Functions::getRequestParameter("controller");
}
$module = ($ReqMod) ? $ReqMod : 'default';
In this style I call the views and actions in classes, like SaveAction()
Using mysql always made it very simple to use this database connection in the models called by the controllers like
public function loadCustomersList($sAdditionalWhere = false)
{
$sQuery = "SELECT * FROM customers WHERE 1 ";
if ($sAdditionalWhere) {
$sQuery .= "AND " . $sAdditionalWhere . " ";
}
$sQuery .= "ORDER BY company";
$sql = mysql_query($sQuery);
while (($customer = mysql_fetch_object($sql)) != false) {
$aCustomers[] = $customer;
}
return $aCustomers;
}
I want to totally refractor this project and use PDO. I tried for the last 4 hours to find a solution, but I can't figure out how to make it work.
I think I don't need an extra dbconnect class since PDO is a class itself, am I right?
In the new index file I tried the following:
$db = new database();
try{
$dbc = new PDO($db->get_DbConSettings());
$dbc->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
}catch(PDOException $e)
{
echo 'Verbindung fehlgeschlagen: '.$e->getMessage();
}
But with this $dbc will not available in controllers or models. It there a way to make it available there? If not, what is the best solution?
Do I have to make a database connection in every model?
An other issue I have with this is:
$db->get_DbConSettings()
in
$dbc = new PDO
gives back
'mysql:host=127.0.0.1; dbname=c1virtbkk', 'root', '123'
($dbc = new PDO('mysql:host=127.0.0.1; dbname=c1virtbkk', 'root', '123');)
I cannot connect to the database. I get the following:
Verbindung fehlgeschlagen: could not find driver
If I don't use $db->get_DbConSettings and put the required information manually in, I don't get any error and can do queries. Any hints?
Help is really appreciated.
Thanks in advance!
Mark
Definitely don't create a new PDO connection in each model. Creating MySQL database connections is fairly quick, but there's still some overhead to doing so. You want to reuse a connection throughout your request. If nothing else, it allows you to share a transaction across multiple models.
Some frameworks store shared resource objects in a "registry" class which is a singleton key-value store. It's not really much more than a global hash array, but making it a class makes the registry itself more easily tested with PHPUnit. See https://framework.zend.com/manual/1.12/en/zend.registry.using.html for an example of a registry.
You're right that PDO is a class, even though it's implemented as a C extension instead of a PHP class. But it's a class, and new PDO(...) returns an object of that class.
One reason to create a db class of your own is to help you in unit-testing, because you could create a mock object for your db class so you can test your other classes (even model classes) without needing a live database connection. Your db class could extend or else contain a PDO object.
Your issue about the error "could not find driver" is probably because the PDO driver for mysql is not installed. PDO is one PHP extension, and then there's a separate extension for each brand of SQL database. You can confirm this with:
$ php -i
...lots of output...
PDO
PDO support => enabled
PDO drivers => mysql, odbc, sqlite
pdo_mysql
PDO Driver for MySQL => enabled
Client API version => mysqlnd 5.0.11-dev - 20120503 - $Id: 76b08b24596e12d4553bd41fc93cccd5bac2fe7a $
...more output for other extensions...
Note that PDO tells me which drivers I have installed: mysql, odbc, and sqlite.
So you need to install pdo_mysql. I'm not sure what OS you're on, but I'm often on CentOS Linux or Ubuntu Linux. The pdo_mysql is available as a separate package via yum or apt.
Re your comment:
Okay, here's an example of a registry:
class registry {
protected static $items = array();
public static get($key) {
return isset(self::$items[$key])?
self::$items[$key] : null;
}
public static set($key, $object) {
self::$items[$key] = $object;
}
}
In your controller initial code, you'd create a database object and store it in the registry:
$projectConfiguration = new projectConfiguration();
$dbconnect = $projectConfiguration->connect(
$projectConfiguration->databaseHost,
$projectConfiguration->databaseName,
$projectConfiguration->databaseUser,
$projectConfiguration->databasePass);
registry::set('db', $dbconnect);
Then in your model class methods (or anywhere you need the database), get the db object from the registry and use it:
public function loadCustomersList($sAdditionalWhere = false)
{
$sQuery = "SELECT * FROM customers WHERE 1 ";
if ($sAdditionalWhere) {
$sQuery .= "AND " . $sAdditionalWhere . " ";
}
$sQuery .= "ORDER BY company";
$db = registry::get('db');
$stmt = $db->query($sQuery);
$aCustomers = $stmt->fetchAll(PDO::FETCH_ASSOC);
return $aCustomers;
}
First up, I apologise if this is a duplicate in any way, but I have been trying many different things from this site for several hours now with no luck. And for the record, I am running OS X 10.11.5.
I have made this simple application using JDBC to connect to a database I created that is stored on my localhost (I am using phpMyAdmin, if that is any help):
#Override
public void actionPerformed(ActionEvent e) {
// The ID to search for
int search = Integer.parseInt(textField.getText());
// Setting up the connection to the database.
String url = "jdbc:mysql://my_ip_address:3306/javaDatabase";
String user = "root"; // root user
String password = ""; // no password
Connection con = null;
PreparedStatement searchID; // I used a prepared statement so I could include user input
ResultSet result = null; // results after SQL execution
try {
con = DriverManager.getConnection(url, user, password); // connect to MySQL
// Creating the prepared statement
searchID = con.prepareStatement("SELECT * FROM Names WHERE ID = ?");
// Setting the parameter to the user input
searchID.setInt(1, search);
result = searchID.executeQuery(); // execute the SQL query
while (result.next()) { // loop until the end of the results
String ID = result.getString("ID");
String FirstName = result.getString("FirstName");
String LastName = result.getString("LastName");
textArea1.setText("ID: " + ID + "\n" +
"First Name: " + FirstName + "\n" +
"Last Name: " + LastName + "\n");
}
} catch(SQLException e1) {
System.out.println("Exception caught " + e1.getMessage());
} finally {
try {
if (result != null) {
result.close();
}
if (con != null) {
con.close();
}
} catch(SQLException e2) {
System.out.println("SQLException caught " + e2.getMessage());
}
}
}
public static void main(String[] args) {
JavaWithSQL newJava = new JavaWithSQL();
}
Now, I am packaging this application up as an executable .JAR file, and want to be able to run it on someone else's computer and have it access the database and return the records.
I have tried instructions from here and here, without any luck. I looked at opening port 3306 on my Mac, but my firewall is off, but that doesn't seem to be the problem. I have also attempted to use GRANT privileges on the database in question using:
GRANT ALL PRIVILEGES ON javaDatabase.* TO '%'#'%' IDENTIFIED BY '';
to no avail. However, it does work on other computers when I explicitly write the computers IP address, like this:
GRANT ALL PRIVILEGES ON javaDatabase.* TO 'root'#'other_computer_ip' IDENTIFIED BY '';
But I need to be able to run it on my lecturer's computer, and in theory, other people's computers, without having to know everyone's IP addresses.
How can I do this? Am I missing something?
EDIT:
Okay, I have run the command
GRANT SELECT ON javaDatabase.* TO 'remoteUser'#'%' IDENTIFIED BY '';
FLUSH PRIVILEGES;
And now it works perfectly on any computer connected to the same network, but I need it to work even if I am connected to a different network.
I really need a pointer on this one.
You are using TO '%'#'%' IDENTIFIED BY '';
I'm not sure if that works. It should address the root user in your case.
GRANT ALL PRIVILEGES ON javaDatabase.* TO 'root'#'%' IDENTIFIED BY '';
I use spring boot + MySQL5 database.
There is a periodic service that runs and need to do the following transaction:
Delete records (with condition)
Insert records
In addition another service does select queries and should see a snapshot of the records without interfering with the delete+insert transactions.
I have the following code:
#Service
public class BulkInsert
{
public static final String DELETE_ALL_ROWS_QUERY = "DELETE FROM GnsEntity where is_synced = true and was_removed = false";
#Inject
private EntityManager entityManager;
#Transactional
public void save(List<GnsEntity> gnsEntityList)
{
Session session = entityManager.unwrap(Session.class);
Query entity = session.createQuery(DELETE_ALL_ROWS_QUERY);
entity.executeUpdate();
for (int i = 0; i < gnsEntityList.size(); ++i)
{
try
{
session.persist(gnsEntityList.get(i));
}
catch(NonUniqueObjectException nonUniEx)
{
}
}
}
}
In general it seems to work good.. though a lot of times there's a deadlock exception and I have no clue why..
Thats why I was wondering if my code is relatively fine?
I get the following errors every now and then:
DEBUG","message":"Creating new transaction with name
[com.ddd.swiss.microservices.gnssynchronizer.BulkInsert.save]:
PROPAGATION_REQUIRED,ISOLATION_DEFAULT","service":"GNSSynchronizer","instanceId":"1","application":"Start","space":"ngampel","class":"org.springframework.orm.jpa.JpaTransactionManager","thread":"pool-3-thread-1","X-B3-TraceId":"5db000bfb3de1a6d49a53edd707419a0","X-B3-SpanId":"49a53edd707419a0"}
{"#timestamp":"2019-10-23T07:27:24.318Z","logLevel":"DEBUG","message":"Opened
new EntityManager
[org.hibernate.jpa.internal.EntityManagerImpl#5a445da1] for JPA
transaction","service":"GNSSynchronizer","instanceId":"1","application":"Start","space":"ngampel","class":"org.springframework.orm.jpa.JpaTransactionManager","thread":"pool-3-thread-1","X-B3-TraceId":"5db000bfb3de1a6d49a53edd707419a0","X-B3-SpanId":"49a53edd707419a0"}
{"#timestamp":"2019-10-23T07:27:24.318Z","logLevel":"DEBUG","message":"begin","service":"GNSSynchronizer","instanceId":"1","application":"Start","space":"ngampel","class":"org.hibernate.engine.transaction.internal.TransactionImpl","thread":"pool-3-thread-1","X-B3-TraceId":"5db000bfb3de1a6d49a53edd707419a0","X-B3-SpanId":"49a53edd707419a0"}
{"#timestamp":"2019-10-23T07:27:24.319Z","logLevel":"DEBUG","message":"Exposing
JPA transaction as JDBC transaction
[org.springframework.orm.jpa.vendor.HibernateJpaDialect$HibernateConnectionHandle#241c36b8]","service":"GNSSynchronizer","instanceId":"1","application":"Start","space":"ngampel","class":"org.springframework.orm.jpa.JpaTransactionManager","thread":"pool-3-thread-1","X-B3-TraceId":"5db000bfb3de1a6d49a53edd707419a0","X-B3-SpanId":"49a53edd707419a0"}
{"#timestamp":"2019-10-23T07:27:24.319Z","logLevel":"DEBUG","message":"delete
from gns_entity where is_synced=1 and
was_removed=0","service":"GNSSynchronizer","instanceId":"1","application":"Start","space":"ngampel","class":"org.hibernate.SQL","thread":"pool-3-thread-1","X-B3-TraceId":"5db000bfb3de1a6d49a53edd707419a0","X-B3-SpanId":"49a53edd707419a0"}
{"#timestamp":"2019-10-23T07:27:25.451Z","logLevel":"DEBUG","message":"could
not execute statement
[n/a]","stackTrace":"com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException:
Deadlock found when trying to get lock; try restarting
transaction\n\tat
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Nativ
Thanks for the help!