I need to validate a workload on a DB used to answer to http api.
In this context, on production, there are a lot of connections opened / closed. For a connection, there are only 2 or 3 small queries launched.. So connection 'activity' (open/close) has to be taken into account in our application.
I need to 'bench' / test the DB without the application stack, so I'd like JMETER to query directly the database like the web service would do..
When using / configuring odbc connection pool through "jdbc connection configuration", I only see the way to define a large pool of connection that will be used, after, to launch queries. That mean... the connections stay alive after playing ThreadGroup scenario, and are reused. In real application, for a scenario, this would make a new connection, and would close this one at the end.
Is there a way to do it (make a new connection for every ThreadGroup run) in JMETER with JDBC 'components' ?
as a workarround, I created a small script and asked jmeter to run it... but it's far more heavier for the server to do it (launch a new process each time to execute the (php) script.. and I couldn't load the server enough by doing it, to reproduce the workload.
JMeter is actually calling Connection.close() function after executing the statement, under the hood the connection is being returned to the pool and it waits for the next thread which requires the connection.
If your application behaviour is the same you don't need to worry about anything. If it's different - you won't get such precise control with the JDBC Connection Configuration and JDBC Request sampler.
If you want to create and destroy connections manually you will have to switch to JSR232 Sampler and implement connection and query logic in Groovy, see Working with a relational database Groovy user manual chapter for more details, code examples, etc.
I am setting up an SSIS project for testing. This project has a package where I manage some connections to SQL Server data bases, for doing this I use Package Configurations. Here I have a configuration that takes four connection strings from a table. Each is being mapped to connection string property of SSIS OLEDB connection (As shown in next image).
When I turn off "Work offline" and test the connection it works, the problem is that when execute the solution it logs some errors that are releated with the incorrect login to the data base.
I tried to do some workaround for this by setting everything manually (without Package Configurations) and changing protection level to "EncryptSensitiveWithPassword" and it worked.
It seems that the password of the connection string is not being saved when being taken from package configurations.
Do SSIS ConnectionManagers attempt to make their connections on package start, or when the connection is first used?
If it's the latter, I can use failure transitions to handle connection failures within the package, but if it's the former then I have to do it externally.
I can assume that they'd design it with the latter, but I can't find it in the documentation anywhere easily searchable/accessible.
It depends and can be controlled.
With the default setting of DelayValidation=false property, SSIS engine at package start tries to connect (validate) all Connection Managers and tasks with metadata and which use Connection Managers like Data Flows etc.
If you set DelayValidation=true on the Connection Manager itself and all relevant tasks, then the first connection on the Connection Manager for the validation would be executed right before the task is executed. This is quite helpful if you use expressions on Connection Manager properties, like dynamic connection string, and before package execution it cannot be evaluated.
In SSIS,it always try to validate connection manager at the package start time.
Once the SSIS package start, it first validate for all parameter, variable and connection manager etc.
Is it possible to cache database connections when using PHP like you would in a J2EE container? If so, how?
There is no connection pooling in php.
mysql_pconnect and connection pooling are two different things.
There are many problems connected with mysql_pconnect and first you should read the manual and carefully use it, but this is not connection pooling.
Connection pooling is a technique where the application server manages the connections. When the application needs a connection it asks the application server for it and the application server returns one of the pooled connections if there is one free.
We can do connection scaling in php for that please go through following link: http://www.oracle.com/technetwork/articles/dsl/white-php-part1-355135.html
So no connection pooling in php.
As Julio said apache releases all resources when the request ends for the current reques. You can use mysql_pconnect but you are limited with that function and you must be very careful. Other choice is to use singleton pattern, but none of this is pooling.
This is a good article: https://blogs.oracle.com/opal/highly-scalable-connection-pooling-in-php
Also read this one http://www.apache2.es/2.2.2/mod/mod_dbd.html
Persistent connections are nothing like connection pooling. A persistent connection in php will only be reused if you make multiple db connects within the same request/script execution context. In most typical web dev scenarios you'll max out your connections way faster if you use mysql_pconnect because your script will have no way to get a reference to any open connections on your next request. The best way to use db connections in php is to make a singleton instance of a db object so that the connection is reused within the context of your script execution. This still incurs at least 1 db connect per request, but it's better than making multiple db connects per reqeust.
There is no real db connection pooling in php due to the nature of php. Php is not an application server that can sit there in between requests and manage references to a pool of open connections, at least not without some kind of major hack. I think in theory you could write an app server in php and run it as a commandline script that would just sit there in the background and keep a bunch of db connections open and pass references to them to your other scripts, but I don't know if that would be possible in practice, how you'd pass the references from your commandline script to other scripts, and I sort of doubt it would perform well even if you could pull it off. Anyway that's mostly speculation. I did just notice the link someone else posted to an apache module to allow connection pooling for prefork servers such as php. Looks interesting:
https://github.com/junamai2000/mod_namy_pool#readme
I suppose you're using mod_php, right?
When a PHP file finishes executing all it's state is killed so there's no way (in PHP code) to do connection pooling. Instead you have to rely on extensions.
You can mysql_pconnect so that your connections won't get closed after the page finishes, that way they get reused in the next request.
This might be all that you need but this isn't the same as connection pooling as there's no way to specify the number of connections to maintain opened.
You can use MySQLi.
For more info, scroll down to Connection pooling section # http://www.php.net/manual/en/mysqli.quickstart.connections.php#example-1622
Note that Connection pooling is also dependent on your server (i.e. Apache httpd) and its configuration.
If an unused persistent connection for a given combination of "host, username, password, socket, port and default database can not be found" in the open connection pool, then only mysqli opens a new connection otherwise it would reuse already open available persistent connections, which is in a way similar to the concept of connection pooling. The use of persistent connections can be enabled and disabled using the PHP directive mysqli.allow_persistent. The total number of connections opened by a script can be limited with mysqli.max_links (this may be interesting to you to address max_user_connections issue hitting hosting server's limit). The maximum number of persistent connections per PHP process can be restricted with mysqli.max_persistent.
In wider programming context, it's a task of web/app server however in this context, it's being handled by mysqli directive of PHP itself in a way supporting connection re-usability. You may also implement a singleton class to get a static instance of connection to reuse just like in Java. Just want to remind that java also doesn't support connection pooling as part of its standard JDBC, they're being different module/layers on top of JDBC drivers.
Coming to PHP, the good thing is that for the common databases in the PHP echosystem it does support Persistent Database Connections which persists the connection for 500 requests (config of max_requests in php.ini) and this avoids creating a new connection in each request. So check it out in docs in detail, it solves most of your challenges. Please note that PHP is not so much sophisticated in terms of extensive multi-threading mechanism and concurrent processing together with powerful asynchronous event handling, when compared to strictly object oriented Java. So in a way it is very less effective for PHP to have such in-built mechanism like pooling.
You cannot instantiate connection pools manually.
But you can use the "built in" connection pooling with the mysql_pconnect function.
I would like to suggest PDO::ATTR_PERSISTENT
Persistent connections are links that do not close when the execution of your script ends. When a persistent connection is requested, PHP checks if there's already an identical persistent connection (that remained open from earlier) - and if it exists, it uses it. If it does not exist, it creates the link.
Connection pooling works at MySQL server side like this.
If persistence connection is enabled into MySQL server config then MySQL keep a connection open and in sleep state after requested client (php script) finises its work and die.
When a 2nd request comes with same credential data (Same User Name, Same Password, Same Connection Parameter, Same Database name, Maybe from same IP, I am not sure about the IP) Then MySQL pool the previous connection from sleep state to active state and let the client use the connection. This helps MySQL to save time for initial resource for connection and reduce the total number of connection.
So the connection pooling option is actually available at MySQL server side. At PHP code end there is no option. mysql_pconnect() is just a wrapper that inform PHP to not send connection close request signal at the end of script run.
For features such as connection pooling - you need to install swoole extension first: https://openswoole.com/
It adds async features to php.
After that its trivial to add mysql and redis connection pooling:
https://github.com/open-smf/connection-pool
Some PHP frameworks come with pooling built-in: https://hyperf.wiki/2.2/#/en/pool
I have an SSIS package that contains a Lookup Transformation. This requires a connection to the database. At the connection manager, when I use the Windows authentication the connection works fine but when I use SQL Server Authentication then I get this error:
[A140_DAMAGETYPETEXT [3799]] Error: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager "servername.Datenbankname.sqlUser" failed with error code 0xC0202009. There may be error messages posted before this with more information on why the AcquireConnection method call failed.
Connection string:
Data Source=TheServerName;User ID=TheSQLUser;Initial Catalog=TheDababase;Provider=SQLNCLI10.1;Persist Security Info=True;Application Name=NameOfThePackage{80F4BFAB-642D-457E-9F47-1E811AFA48D0}ServerName.TheDababase.TheSQLUser;
Cause of the issue:
Your connection string is missing the Password attribute. That is the reason why the connection manager is failing to connect to the database instance.
Possible option to fix:
When you create the connection manager on SSIS, make sure to check the Save my password box so that the Password is retained in the connection string. If you do not check this box, the value will not be retained and you have to key in the value.
Package configuration:
If you are using package configurations, adding / updating the connection manager will cause the Password attribute to drop from the connection string property. I usually use database based package configuration. So, I use a query to update the connection string in the package configuration table every time I update the package configurations.
Recommendation:
I prefer to use Windows authentication so that the password to connect to the database is not exposed. I feel that Windows Authentication is little more secure than dealing with SQL Server Authentication.
here was the solution for me: click
the thread from Raj_007 was the rescue. I use a sequece container and the TransactionOption was setting to required.
I changed the TransactionOption to supported and it worked for me.