how to add more storage plugins programatically in apache drill? - apache-drill

Question

I tried drill JDBC driver to query programmatically.
Useful portion of code:
Connection conn = new Driver().connect("jdbc:drill:zk=local", getDefaultProperties());
Statement stmt = conn.createStatement();
ResultSet rs = stmt.executeQuery("show databases");
while (rs.next())
{
String SCHEMA_NAME = rs.getString("SCHEMA_NAME");
System.out.println(SCHEMA_NAME);
}
public static Properties getDefaultProperties()
{
final Properties properties = new Properties();
properties.setProperty(ExecConstants.HTTP_ENABLE, "false");
return properties;
}
Everything worked fine till I have cp & dfs storage plugin. Output of above query:
INFORMATION_SCHEMA
cp.default
dfs.default
dfs.root
dfs.tmp
sys
But when I added mongo as storage plugin with configuration:
{
"type": "mongo",
"connection": "mongodb://localhost:27017/",
"enabled": false
}
I got following exception:
java.sql.SQLException: Failure in starting embedded Drillbit: java.lang.RuntimeException: Unable to deserialize "/tmp/drill/sys.storage_plugins/mongo.sys.drill"
at org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:109)
at org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:66)
at org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
at net.hydromatic.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:126)
at org.apache.drill.jdbc.Driver.connect(Driver.java:78)
at com.mkyong.App.main(App.java:37)
Caused by: java.lang.RuntimeException: Unable to deserialize "/tmp/drill/sys.storage_plugins/mongo.sys.drill"
at org.apache.drill.exec.store.sys.local.FilePStore.get(FilePStore.java:140)
at org.apache.drill.exec.store.sys.local.FilePStore$Iter$DeferredEntry.getValue(FilePStore.java:219)
at org.apache.drill.exec.store.StoragePluginRegistry.createPlugins(StoragePluginRegistry.java:168)
at org.apache.drill.exec.store.StoragePluginRegistry.init(StoragePluginRegistry.java:132)
at org.apache.drill.exec.server.Drillbit.run(Drillbit.java:244)
at org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:100)
... 5 more
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Could not resolve type id 'mongo' into a subtype of [simple type, class org.apache.drill.common.logical.StoragePluginConfig]
at [Source: [B#21318883; line: 2, column: 3]
at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)
at com.fasterxml.jackson.databind.DeserializationContext.unknownTypeException(DeserializationContext.java:849)
at com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:167)
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:99)
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84)
at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132)
at com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:41)
at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:1269)
at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:912)
at org.apache.drill.exec.store.sys.serialize.JacksonSerializer.deserialize(JacksonSerializer.java:44)
at org.apache.drill.exec.store.sys.local.FilePStore.get(FilePStore.java:138)
... 10 more
Also, how can I add plugin configuration programmatically?
Edit: Similar behaviour for hive.

Answer 1

Drill provides a REST API. I've used the curl command shown in the docs:
curl -X POST -/json" -d '{"name":"myplugin", "config": {"type": "file", "enabled": false, "connection": "file:///", "workspaces": { "root": { "location": "/", "writable": false, "defaultInputFormat": null}}, "formats": null}}' http://localhost:8047/storage/myplugin.json