Configure Spring Data Couchbase on a cluster host address - couchbase

My data people gave me the http://127.0.0.1:8091/pools url to connect to our Couchbase server and I've been told the pools suffix is the address to all the nodes in the cluster.
I'm using Spring 4.2.0.RELEASE with spring-data-couchbase 2.0.0.M1 against Couchbase 2.5.1 enterprise edition (build-1083)
Now, if I add the above url as is into the getBootstrapHosts list:
#Override
protected List<String> getBootstrapHosts() {
return Collections.singletonList(couchbaseProperties.getHost());
}
I get a number format exception on the 8091/pools value.
But when using the http://127.0.0.1:8091 url I get an invalid password exception.
I reckon the first url is to be used, but not in the way I went for.
There is probably a method I should override in the AbstractCouchbaseConfiguration class, but looking at the source code didn't really enlighten me.
Here is the Couchbase configuration class.
#Configuration
#EnableCouchbaseRepositories(basePackages = { "com.thalasoft.data.couchbase.repository" })
#ComponentScan(nameGenerator = PackageBeanNameGenerator.class, basePackages = { "com.thalasoft.data.couchbase.config" })
#EnableTransactionManagement
public class CouchbaseConfiguration extends AbstractCouchbaseConfiguration {
private static Logger logger = LoggerFactory.getLogger(CouchbaseConfiguration.class);
#Autowired
private CouchbaseProperties couchbaseProperties;
#Override
protected List<String> getBootstrapHosts() {
return Collections.singletonList(couchbaseProperties.getHost());
}
#Override
protected String getBucketName() {
return couchbaseProperties.getBucketName();
}
#Override
protected String getBucketPassword() {
return couchbaseProperties.getBucketPassword();
}
#Bean
public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
#Bean
public LocalValidatorFactoryBean validator() {
return new LocalValidatorFactoryBean();
}
#Bean
public ValidatingCouchbaseEventListener validationEventListener() {
return new ValidatingCouchbaseEventListener(validator());
}
}

The fact that your database administrators gave you 127.0.0.1 as the adress to connect to seem strange, but indeed could be valid if one node of the cluster is running colocated with the client code...
This url-based syntax was the one used for the 1.4.x generation of SDK, and configuration is indeed a bit different in 2.x (reflecting the evolution of the Couchbase SDK between 1.4.x and 2.x): you just need to provide the hostname or ip of each node to bootstrap from, in a list.
You should try with just "127.0.0.1". It is possible also that you need to specify a bucket name and/or a password (ask your administrator). The defaults used by Spring Data Couchbase for each is "default" and "" (empty password), but you can override the getBucketName() and getBucketPassword() methods from AbsctractCouchbaseConfiguration to change that.
PS: the Spring Data Couchbase documentation is available here

Related

Create Multipe Jdbctemplate at runtime pointing to diffrent databases in runtime

Spring boot Is there any way to create Jdbctemplate at runtime pointing to the different MySQL databases?
Runtime I do have all the required information.
The database structure is the same for all the clients but the instance is different.
I also tried with How to create multiple Jdbctemplate beans, pointing to different Mysql databases, in runtime?
but not able to find any solution.
I guess this can be done like this:
#Component
public class JdbcTemplateProvider {
private final Map<String, DataSource> databaseNameToDataSourceMap;
private final JdbcProperties properties;
public JdbcTemplateProvider(List<DataSource> dataSources, JdbcProperties properties) {
this.databaseNameToDataSourceMap = this.buildDatabaseToDataSourceMap(dataSources);
this.properties = properties;
}
public JdbcTemplate byDatabaseName(String database) {
DataSource dataSource = this.databaseNameToDataSourceMap.get(database);
return buildJdbcTemplate(dataSource);
}
private Map<String, DataSource> buildDatabaseToDataSourceMap(List<DataSource> dataSources) {
return new ConcurrentHashMap<>();
}
private JdbcTemplate buildJdbcTemplate(DataSource dataSource) {
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
JdbcProperties.Template template = properties.getTemplate();
jdbcTemplate.setFetchSize(template.getFetchSize());
jdbcTemplate.setMaxRows(template.getMaxRows());
if (template.getQueryTimeout() != null) {
jdbcTemplate.setQueryTimeout((int) template.getQueryTimeout().getSeconds());
}
return jdbcTemplate;
}
}
If this question was if Spring Boot can natively handle this, as far as I know there may not be any built in solution for this.
org.springframework.boot.autoconfigure.jdbc.JdbcProperties reads properties values with prefix spring.jdbc.If it is available in your classpath, this will be able to configure newly built JdbcTemplate instances as per configuration.
You will have build your own logic though to build map of database name to datasource.

Integration multi node Couchbase with Springboot

I am new to Couchbase and integrating local Couchbase server from my Springboot application, using Couchbase v6.6 and spring-data-couchbase v 4.0.5
I initially had single node running and my application was able to insert documents using CrudRepository.
I recently configured multiple nodes on my localhost using Docker images, but now my Springboot application doesn’t work as expected.
Relevant section of my logs attached, according to which it looks like the service connects to all 3 nodes, but then immediately disconnects for some reason!
My config class looks like this…
public class CouchbaseConfig extends AbstractCouchbaseConfiguration {
#Override
public String getConnectionString() {
return "127.0.0.1";
}
#Override
public String getUserName() {
return "Administrator";
}
#Override
public String getPassword() {
return "password";
}
#Override
public String getBucketName() {
return "bucket";
}
}
Any helpful tips would be appreciated.
Many Thanks

How to connect to multiple MySQL databases as per the header in REST API request

I'm creating a multi tenant spring boot - JPA application.
In this application, I want to connect to MySQL Databases using DB name which is sent through API request as header.
I checked many multi tenant project samples online but still can't figure out a solution.
Can anyone suggest me a way to do this?
You can use AbstractRoutingDataSource to achieve this. AbstractRoutingDataSource requires information to know which actual DataSource to route to(referred to as Context), which is provided by determineCurrentLookupKey() method. Using example from here.
Define Context like:
public enum ClientDatabase {
CLIENT_A, CLIENT_B
}
Then you need to define Context Holder which will be used in determineCurrentLookupKey()
public class ClientDatabaseContextHolder {
private static ThreadLocal<ClientDatabase> CONTEXT = new ThreadLocal<>();
public static void set(ClientDatabase clientDatabase) {
Assert.notNull(clientDatabase, "clientDatabase cannot be null");
CONTEXT.set(clientDatabase);
}
public static ClientDatabase getClientDatabase() {
return CONTEXT.get();
}
public static void clear() {
CONTEXT.remove();
}
}
Then you can extend AbstractRoutingDataSource like below:
public class ClientDataSourceRouter extends AbstractRoutingDataSource {
#Override
protected Object determineCurrentLookupKey() {
return ClientDatabaseContextHolder.getClientDatabase();
}
}
Finally, DataSource bean configuration:
#Bean
public DataSource clientDatasource() {
Map<Object, Object> targetDataSources = new HashMap<>();
DataSource clientADatasource = clientADatasource();
DataSource clientBDatasource = clientBDatasource();
targetDataSources.put(ClientDatabase.CLIENT_A,
clientADatasource);
targetDataSources.put(ClientDatabase.CLIENT_B,
clientBDatasource);
ClientDataSourceRouter clientRoutingDatasource
= new ClientDataSourceRouter();
clientRoutingDatasource.setTargetDataSources(targetDataSources);
clientRoutingDatasource.setDefaultTargetDataSource(clientADatasource);
return clientRoutingDatasource;
}
https://github.com/wmeints/spring-multi-tenant-demo
Following this logic, I can solve it now. Some of the versions need to be upgraded and the codes as well.
Spring Boot version have changed.
org.springframework.boot
spring-boot-starter-parent
2.1.0.RELEASE
Mysql version has been removed.
And some small changed in MultitenantConfiguration.java
#Configuration
public class MultitenantConfiguration {
#Autowired
private DataSourceProperties properties;
/**
* Defines the data source for the application
* #return
*/
#Bean
#ConfigurationProperties(
prefix = "spring.datasource"
)
public DataSource dataSource() {
File[] files = Paths.get("tenants").toFile().listFiles();
Map<Object,Object> resolvedDataSources = new HashMap<>();
if(files != null) {
for (File propertyFile : files) {
Properties tenantProperties = new Properties();
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create(this.getClass().getClassLoader());
try {
tenantProperties.load(new FileInputStream(propertyFile));
String tenantId = tenantProperties.getProperty("name");
dataSourceBuilder.driverClassName(properties.getDriverClassName())
.url(tenantProperties.getProperty("datasource.url"))
.username(tenantProperties.getProperty("datasource.username"))
.password(tenantProperties.getProperty("datasource.password"));
if (properties.getType() != null) {
dataSourceBuilder.type(properties.getType());
}
resolvedDataSources.put(tenantId, dataSourceBuilder.build());
} catch (IOException e) {
e.printStackTrace();
return null;
}
}
}
// Create the final multi-tenant source.
// It needs a default database to connect to.
// Make sure that the default database is actually an empty tenant database.
// Don't use that for a regular tenant if you want things to be safe!
MultitenantDataSource dataSource = new MultitenantDataSource();
dataSource.setDefaultTargetDataSource(defaultDataSource());
dataSource.setTargetDataSources(resolvedDataSources);
// Call this to finalize the initialization of the data source.
dataSource.afterPropertiesSet();
return dataSource;
}
/**
* Creates the default data source for the application
* #return
*/
private DataSource defaultDataSource() {
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create(this.getClass().getClassLoader())
.driverClassName(properties.getDriverClassName())
.url(properties.getUrl())
.username(properties.getUsername())
.password(properties.getPassword());
if(properties.getType() != null) {
dataSourceBuilder.type(properties.getType());
}
return dataSourceBuilder.build();
}
}
This change is here due to the DataSourceBuilder has been moved to another path and its constructor has been changed.
Also changed the MySQL driver class name in application.properties like this
spring.datasource.driver-class-name=com.mysql.jdbc.Driver

How to correctly load Firebase ServiceAccount json resource with Spring MVC?

I'm trying to connect my Spring MVC (not Spring Boot) application to Firebase. My application's folder structure looks like this:
folder structure
The problem is that I don't know where to place the api key json file, how to load the resource, and the correct order of the method calls.
I tried loading the resource the way shown below. Before that I also tried using ClassLoader to load it from the WEB-INF folder and it worked, but changed the code and kept receiving NullPointer Exception (why not FileNotFound Exception?) for the InputStream and couldn't restore the previous state.
With the current state I keep receiving FileNotFound Exception as I'm am not able to load the resource no matter how much I googled "Spring MVC load resource" and as I checked the debugger the service account's "init" method with #PostConstruct isn't running at starting the server.
I understand that I should be able to load the resource and call the "init" method in order to make it work. (I suppose it's enough to call it once after creating the bean and before using firebase methods) But I just couldn't come up with a working implementation.
I used examples from here:
https://github.com/savicprvoslav/Spring-Boot-starter
(Bottom of the Page)
My Controller Class:
#Controller
#RequestMapping("/firebase")
public class FirebaseController {
#Autowired
private FirebaseService firebaseService;
#GetMapping(value="/upload/maincategories")
public void uploadMainRecordCategories() {
firebaseService.uploadMainRecordCategories();
}
My Service Class:
#Service
public class FirebaseServiceBean implements FirebaseService {
#Value("/api.json")
Resource apiKey;
#Override
public void uploadMainRecordCategories() {
// do something
}
#PostConstruct
public void init() {
try (InputStream serviceAccount = apiKey.getInputStream()) {
FirebaseOptions options = new FirebaseOptions.Builder()
.setCredentials(GoogleCredentials.fromStream(serviceAccount))
.setDatabaseUrl(FirebaseStringValue.DB_URL).build();
FirebaseApp.initializeApp(options);
} catch (IOException e) {
e.printStackTrace();
}
}
}
how about saving value in a spring property and using #Value("${firebase.apiKey}")?
Alternatively, save path to file in property and reference that in #Value()
#Value("${service.account.path}")
private String serviceAccountPath;
In application.properties:
service.account.path = /path/to/service-account.json
then config code:
private String getAccessToken() throws IOException {
GoogleCredential googleCredential = GoogleCredential
.fromStream(getServiceAccountInputStream())
.createScoped(Collections.singletonList("https://www.googleapis.com/auth/firebase.messaging"));
googleCredential.refreshToken();
return googleCredential.getAccessToken();
}
private InputStream getServiceAccountInputStream() {
File file = new File(serviceAccountPath);
try {
return new FileInputStream(file);
} catch (FileNotFoundException e) {
throw new RuntimeException("Couldn't find service-account.json");
}
}

How to use annotation based configuration to configure multiple buckets in Couchbase?

In the Couchbase documentation the below is given as an example to configure the environment. What would be done for having more than one bucket ?
#Configuration
public class Config extends AbstractCouchbaseConfiguration {
#Override
protected List<String> getBootstrapHosts() {
return Collections.singletonList("127.0.0.1");
}
#Override
protected String getBucketName() {
return "beer-sample";
}
#Override
protected String getBucketPassword() {
return "";
}
}
For multiple buckets in 2.0.x branch, the way it currently works is that you have to instantiate a second Bucket bean and the associated CouchbaseTemplate (that's the hardest part):
//we want all User objects to be stored in a second bucket
//let's define the bucket reference...
#Bean
public Bucket userBucket() {
return couchbaseCluster().openBucket("users", "");
}
//... then the template (inspired by couchbaseTemplate() method)...
#Bean
public CouchbaseTemplate userTemplate() {
CouchbaseTemplate template = new CouchbaseTemplate(
couchbaseClusterInfo(), //reuse the default bean
userBucket(), //the bucket is non-default
mappingCouchbaseConverter(), translationService() //default beans here as well
);
template.setDefaultConsistency(getDefaultConsistency());
return template;
}
After that, you'll probably also want some of your repositories to use this second template (and bucket). There's currently an implementation for that as well (RepositoryOperationsMapping), but it's probably going to change a bit until upcoming 2.0.0-RC so I won't go into details there.