I am new to Couchbase and integrating local Couchbase server from my Springboot application, using Couchbase v6.6 and spring-data-couchbase v 4.0.5
I initially had single node running and my application was able to insert documents using CrudRepository.
I recently configured multiple nodes on my localhost using Docker images, but now my Springboot application doesn’t work as expected.
Relevant section of my logs attached, according to which it looks like the service connects to all 3 nodes, but then immediately disconnects for some reason!
My config class looks like this…
public class CouchbaseConfig extends AbstractCouchbaseConfiguration {
#Override
public String getConnectionString() {
return "127.0.0.1";
}
#Override
public String getUserName() {
return "Administrator";
}
#Override
public String getPassword() {
return "password";
}
#Override
public String getBucketName() {
return "bucket";
}
}
Any helpful tips would be appreciated.
Many Thanks
Related
Spring boot Is there any way to create Jdbctemplate at runtime pointing to the different MySQL databases?
Runtime I do have all the required information.
The database structure is the same for all the clients but the instance is different.
I also tried with How to create multiple Jdbctemplate beans, pointing to different Mysql databases, in runtime?
but not able to find any solution.
I guess this can be done like this:
#Component
public class JdbcTemplateProvider {
private final Map<String, DataSource> databaseNameToDataSourceMap;
private final JdbcProperties properties;
public JdbcTemplateProvider(List<DataSource> dataSources, JdbcProperties properties) {
this.databaseNameToDataSourceMap = this.buildDatabaseToDataSourceMap(dataSources);
this.properties = properties;
}
public JdbcTemplate byDatabaseName(String database) {
DataSource dataSource = this.databaseNameToDataSourceMap.get(database);
return buildJdbcTemplate(dataSource);
}
private Map<String, DataSource> buildDatabaseToDataSourceMap(List<DataSource> dataSources) {
return new ConcurrentHashMap<>();
}
private JdbcTemplate buildJdbcTemplate(DataSource dataSource) {
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
JdbcProperties.Template template = properties.getTemplate();
jdbcTemplate.setFetchSize(template.getFetchSize());
jdbcTemplate.setMaxRows(template.getMaxRows());
if (template.getQueryTimeout() != null) {
jdbcTemplate.setQueryTimeout((int) template.getQueryTimeout().getSeconds());
}
return jdbcTemplate;
}
}
If this question was if Spring Boot can natively handle this, as far as I know there may not be any built in solution for this.
org.springframework.boot.autoconfigure.jdbc.JdbcProperties reads properties values with prefix spring.jdbc.If it is available in your classpath, this will be able to configure newly built JdbcTemplate instances as per configuration.
You will have build your own logic though to build map of database name to datasource.
I'm creating a multi tenant spring boot - JPA application.
In this application, I want to connect to MySQL Databases using DB name which is sent through API request as header.
I checked many multi tenant project samples online but still can't figure out a solution.
Can anyone suggest me a way to do this?
You can use AbstractRoutingDataSource to achieve this. AbstractRoutingDataSource requires information to know which actual DataSource to route to(referred to as Context), which is provided by determineCurrentLookupKey() method. Using example from here.
Define Context like:
public enum ClientDatabase {
CLIENT_A, CLIENT_B
}
Then you need to define Context Holder which will be used in determineCurrentLookupKey()
public class ClientDatabaseContextHolder {
private static ThreadLocal<ClientDatabase> CONTEXT = new ThreadLocal<>();
public static void set(ClientDatabase clientDatabase) {
Assert.notNull(clientDatabase, "clientDatabase cannot be null");
CONTEXT.set(clientDatabase);
}
public static ClientDatabase getClientDatabase() {
return CONTEXT.get();
}
public static void clear() {
CONTEXT.remove();
}
}
Then you can extend AbstractRoutingDataSource like below:
public class ClientDataSourceRouter extends AbstractRoutingDataSource {
#Override
protected Object determineCurrentLookupKey() {
return ClientDatabaseContextHolder.getClientDatabase();
}
}
Finally, DataSource bean configuration:
#Bean
public DataSource clientDatasource() {
Map<Object, Object> targetDataSources = new HashMap<>();
DataSource clientADatasource = clientADatasource();
DataSource clientBDatasource = clientBDatasource();
targetDataSources.put(ClientDatabase.CLIENT_A,
clientADatasource);
targetDataSources.put(ClientDatabase.CLIENT_B,
clientBDatasource);
ClientDataSourceRouter clientRoutingDatasource
= new ClientDataSourceRouter();
clientRoutingDatasource.setTargetDataSources(targetDataSources);
clientRoutingDatasource.setDefaultTargetDataSource(clientADatasource);
return clientRoutingDatasource;
}
https://github.com/wmeints/spring-multi-tenant-demo
Following this logic, I can solve it now. Some of the versions need to be upgraded and the codes as well.
Spring Boot version have changed.
org.springframework.boot
spring-boot-starter-parent
2.1.0.RELEASE
Mysql version has been removed.
And some small changed in MultitenantConfiguration.java
#Configuration
public class MultitenantConfiguration {
#Autowired
private DataSourceProperties properties;
/**
* Defines the data source for the application
* #return
*/
#Bean
#ConfigurationProperties(
prefix = "spring.datasource"
)
public DataSource dataSource() {
File[] files = Paths.get("tenants").toFile().listFiles();
Map<Object,Object> resolvedDataSources = new HashMap<>();
if(files != null) {
for (File propertyFile : files) {
Properties tenantProperties = new Properties();
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create(this.getClass().getClassLoader());
try {
tenantProperties.load(new FileInputStream(propertyFile));
String tenantId = tenantProperties.getProperty("name");
dataSourceBuilder.driverClassName(properties.getDriverClassName())
.url(tenantProperties.getProperty("datasource.url"))
.username(tenantProperties.getProperty("datasource.username"))
.password(tenantProperties.getProperty("datasource.password"));
if (properties.getType() != null) {
dataSourceBuilder.type(properties.getType());
}
resolvedDataSources.put(tenantId, dataSourceBuilder.build());
} catch (IOException e) {
e.printStackTrace();
return null;
}
}
}
// Create the final multi-tenant source.
// It needs a default database to connect to.
// Make sure that the default database is actually an empty tenant database.
// Don't use that for a regular tenant if you want things to be safe!
MultitenantDataSource dataSource = new MultitenantDataSource();
dataSource.setDefaultTargetDataSource(defaultDataSource());
dataSource.setTargetDataSources(resolvedDataSources);
// Call this to finalize the initialization of the data source.
dataSource.afterPropertiesSet();
return dataSource;
}
/**
* Creates the default data source for the application
* #return
*/
private DataSource defaultDataSource() {
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create(this.getClass().getClassLoader())
.driverClassName(properties.getDriverClassName())
.url(properties.getUrl())
.username(properties.getUsername())
.password(properties.getPassword());
if(properties.getType() != null) {
dataSourceBuilder.type(properties.getType());
}
return dataSourceBuilder.build();
}
}
This change is here due to the DataSourceBuilder has been moved to another path and its constructor has been changed.
Also changed the MySQL driver class name in application.properties like this
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
I am new to Java and Playframework as well. I have play running. But am unable to retrieve data from oracle. I added the /lib folder and added the ojdbc6.jar and play deployed successfully. I know play is connected to my DB as when I had the incorrect config it complained. The config is below in the application.conf
db.default.driver=oracle.jdbc.driver.OracleDriver
db.default.url="jdbc:oracle:thin:#hostname:1521:SOMESID"
db.default.user=someuser
db.default.password=somepassword
All the above seems ok. Now in the application.java I am trying to fetch data from DB. I do not want to use Hibernate or any ORM layers. The application.java is below. Added the method getMetaData to talk to Oracle but does not work. Perhaps I have not written the method properly or not imported the relevant libraries. A working .java file would be very helpful. If I can manage to retrieve data form oracle great otherwise hope I don't have to go the spring framework route. Any help would be greatly appreciated. Thanks
package controllers;
import play.*;
import play.mvc.*;
import views.html.*;
import play.db.*;
public class Application extends Controller {
public static Result index() {
//return ok(index.render("Hello Worldxxx"));
//me: the below is the default return
return redirect(routes.Application.tasks());
}
public static Result tasks() {
return TODO;
}
public static Result newTask() {
return TODO;
}
public static Result deleteTask(Long id) {
return TODO;
}
public static Result getMetaData() {
Connection connection = DB.getConnection();
ResultSet resultSet = connection.prepareStatement("SELECT * FROM sometable").executeQuery();
metaData = resultSet.getMetaData();
connection.close();
return ok(metaData.toString());
}
}
My data people gave me the http://127.0.0.1:8091/pools url to connect to our Couchbase server and I've been told the pools suffix is the address to all the nodes in the cluster.
I'm using Spring 4.2.0.RELEASE with spring-data-couchbase 2.0.0.M1 against Couchbase 2.5.1 enterprise edition (build-1083)
Now, if I add the above url as is into the getBootstrapHosts list:
#Override
protected List<String> getBootstrapHosts() {
return Collections.singletonList(couchbaseProperties.getHost());
}
I get a number format exception on the 8091/pools value.
But when using the http://127.0.0.1:8091 url I get an invalid password exception.
I reckon the first url is to be used, but not in the way I went for.
There is probably a method I should override in the AbstractCouchbaseConfiguration class, but looking at the source code didn't really enlighten me.
Here is the Couchbase configuration class.
#Configuration
#EnableCouchbaseRepositories(basePackages = { "com.thalasoft.data.couchbase.repository" })
#ComponentScan(nameGenerator = PackageBeanNameGenerator.class, basePackages = { "com.thalasoft.data.couchbase.config" })
#EnableTransactionManagement
public class CouchbaseConfiguration extends AbstractCouchbaseConfiguration {
private static Logger logger = LoggerFactory.getLogger(CouchbaseConfiguration.class);
#Autowired
private CouchbaseProperties couchbaseProperties;
#Override
protected List<String> getBootstrapHosts() {
return Collections.singletonList(couchbaseProperties.getHost());
}
#Override
protected String getBucketName() {
return couchbaseProperties.getBucketName();
}
#Override
protected String getBucketPassword() {
return couchbaseProperties.getBucketPassword();
}
#Bean
public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
#Bean
public LocalValidatorFactoryBean validator() {
return new LocalValidatorFactoryBean();
}
#Bean
public ValidatingCouchbaseEventListener validationEventListener() {
return new ValidatingCouchbaseEventListener(validator());
}
}
The fact that your database administrators gave you 127.0.0.1 as the adress to connect to seem strange, but indeed could be valid if one node of the cluster is running colocated with the client code...
This url-based syntax was the one used for the 1.4.x generation of SDK, and configuration is indeed a bit different in 2.x (reflecting the evolution of the Couchbase SDK between 1.4.x and 2.x): you just need to provide the hostname or ip of each node to bootstrap from, in a list.
You should try with just "127.0.0.1". It is possible also that you need to specify a bucket name and/or a password (ask your administrator). The defaults used by Spring Data Couchbase for each is "default" and "" (empty password), but you can override the getBucketName() and getBucketPassword() methods from AbsctractCouchbaseConfiguration to change that.
PS: the Spring Data Couchbase documentation is available here
I'm looking at MassTransit as a ServiceBus implementation to use in a web project.
I am playing with the Request/Response pattern and am seeing a long delay between the consumer receiving the message and responding, and the request publisher handling the response; sometimes, it seems like the response is never going to come through (having left it running for 10 minutes, the response has still not come through). the only times that I have seen the handle delegate get called with the response is after a 30 second timeout period and the timeout exception being thrown; in this situation, the breakpoint set on the handler delegate is hit.
The setup is a standard affair - I have a web app that is publishing requests, a console app that is consuming requests and sending responses, for the web app to handle the responses in the callback.
I'm using Castle Windsor, and the container is initialized in the web project using WebActivator:
[assembly: WebActivator.PreApplicationStartMethod(typeof(BootStrapper), "PreStart")]
[assembly: WebActivator.PostApplicationStartMethod(typeof(BootStrapper), "PostStart")]
[assembly: WebActivator.ApplicationShutdownMethodAttribute(typeof(BootStrapper), "Stop")]
namespace Web.App_Start
{
public static class BootStrapper
{
internal static IWindsorContainer Container { get; private set; }
public static void PreStart()
{
Container = new WindsorContainer().Install(FromAssembly.This());
}
public static void PostStart()
{
FilterConfig.RegisterGlobalFilters(GlobalFilters.Filters);
RouteConfig.RegisterRoutes(RouteTable.Routes);
BundleConfig.RegisterBundles(BundleTable.Bundles);
ApiConfig.Configure(Container);
MvcConfig.Configure(Container);
}
public static void Stop()
{
if (Container != null)
Container.Dispose();
}
}
}
In the web app project (an ASP.NET Web API project), the WindsorInstaller for MassTransit looks like
public class MassTransitInstaller : IWindsorInstaller
{
public void Install(IWindsorContainer container, IConfigurationStore store)
{
container.Register(AllTypes.FromThisAssembly().BasedOn<IConsumer>());
var bus = ServiceBusFactory.New(configurator =>
{
configurator.UseMsmq();
configurator.VerifyMsmqConfiguration();
configurator.UseMulticastSubscriptionClient();
configurator.ReceiveFrom("msmq://localhost/web");
configurator.EnableMessageTracing();
configurator.Subscribe(x => x.LoadFrom(container));
});
container.Register(Component.For<IServiceBus>().Instance(bus));
}
}
In the console app project, the WindsorInstaller looks like
public class MassTransitInstaller : IWindsorInstaller
{
public void Install(IWindsorContainer container, IConfigurationStore store)
{
container.Register(AllTypes.FromAssemblyContaining<BasicRequestCommandHandler>().BasedOn<IConsumer>());
var bus = ServiceBusFactory.New(configurator =>
{
configurator.UseMsmq();
configurator.VerifyMsmqConfiguration();
configurator.UseMulticastSubscriptionClient();
configurator.ReceiveFrom("msmq://localhost/console");
configurator.Subscribe(x => x.LoadFrom(container));
});
container.Register(Component.For<IServiceBus>().Instance(bus));
}
}
I have an ApiController with the following GET action method
public class ExampleController : ApiController
{
private readonly IServiceBus _bus;
public HelloController(IServiceBus bus)
{
_bus = bus;
}
// GET api/hello?text={some text}
public Task<IBasicResponseCommand> Get(string text)
{
var command = new BasicRequestCommand {Text = text};
var tcs = new TaskCompletionSource<IBasicResponseCommand>();
_bus.PublishRequest(command, c =>
{
c.Handle<IBasicResponseCommand>(r =>
{
tcs.SetResult(r);
});
});
return tcs.Task;
}
}
BasicRequestCommand and BasicResponseCommand look like so
public interface IBasicRequestCommand
{
Guid CorrelationId { get; set; }
string Text { get; set; }
}
public class BasicRequestCommand :
CorrelatedBy<Guid>, IBasicRequestCommand
{
public Guid CorrelationId { get; set; }
public string Text { get; set; }
public BasicRequestCommand()
{
CorrelationId = Guid.NewGuid();
}
}
public interface IBasicResponseCommand
{
Guid CorrelationId { get; set; }
string Text { get; set; }
}
public class BasicResponseCommand :
CorrelatedBy<Guid>, IBasicResponseCommand
{
public Guid CorrelationId { get; set; }
public string Text { get; set; }
}
And the handler responding to the BasicRequestCommand in the console app:
public class BasicRequestCommandHandler : Consumes<IBasicRequestCommand>.Context
{
public void Consume(IConsumeContext<IBasicRequestCommand> context)
{
Console.Out.WriteLine("received message text " + context.Message.Text);
context.Respond(new BasicResponseCommand { Text = "Hello " + context.Message.Text, CorrelationId = context.Message.CorrelationId });
}
}
I was anticipating with all of this running locally that the request/response would be in the order of a few seconds at most. Am I missing something in configuration?
In addition, I wanted to hook MassTransit up to log4net. I am using Windsor's log4net logging facility and have a log4net section in web.config. This is all working fine for ILogger implementations provided by Windsor (and also for NHibernate logging), but it's not clear from the documentation how to configure MassTransit to use this for logging. Any ideas?
Just as Andrei Volkov and Chris Patterson were discussing on MassTransit google group, it seems that this issue stems from switching MassTransit to using SynchronizationContext, which for some reason does not work as expected.
For the time being one workaround seems to be transitioning to async MassTransit requests, or going back to v2.1.1 that does not use the offending SynchronizationContext.
(Will posts updates on this issue here for posterity if noone else does that first.)
The response timeout issue for Request/Response in ASP.NET is fixed in version 2.6.2.
https://groups.google.com/d/topic/masstransit-discuss/oC1FOe6KsAU/discussion
As you're using the MultiCastSubscriptionClient, you must call SetNetwork(NETWORK_KEY) on each machine (using the same value for NETWORK_KEY). Also, all participating machines need to be on the same subnet - see the documentation at http://masstransit.readthedocs.org/en/latest/overview/subscriptions.html#msmq-multicast
For hooking up log4net, it depends what version you're using, but in the latest versions you include the MassTransit.Log4NetIntegration assembly and then call cfg.UseLog4Net(); in your service bus configuration.
If you're still stuck, you could ask the MT mailing list at https://groups.google.com/forum/?fromgroups#!forum/masstransit-discuss