How can I use drill-embed with file extensions different from the default ones? - apache-drill

Question

I'm trying to test apache drill via drill-embed but all my json files are jsonline files with the jl.gz file extension.
If I rename them to json.gz it works but this is undesirable in my case.
How can I tell drill that jl.gz files are actually json?
PS: I tried adding a bootstrap-storage-plugins.json to $CP but drill-embed doesn't seem to read it.

Answer 1

Yes, do not use bootstrapping. That is only for distributed environments and using the Web Console or REST API is recommended. It probably goes without saying that the gz file must be zipped, not an unzipped JSON file with a gz extension. Create a new storage plugin configuration, for example myplugin, based on the default dfs storage plugin.
Start the Drill shell and go to http://<IP address or host name>:8047. Select Storage in the toolbar. The dfs storage plugin configuration appears in the list of default configurations.
On the Storage tab, under Enabled Storage Plugins, click UPDATE to copy the dfs storage plugin configuration.
The configuration of the plugin appears.
Copy the configuration and go back (just cancel the configuration).
On the Storage tab, enter a name in New Storage Plugin. For example, enter myplugin. Each configuration registered with Drill must have a distinct name. Names are case-sensitive.
Click CREATE.
In Configuration, in the formats section, change the json format to specify extensions: "gz"
"json": {
"type": "json",
"extensions": [
"gz"
]
},
Click CREATE.
Now, in the Drill shell, you can query the JSON file named something.gz:
use myplugin;
select * from `/Users/me/donuts.gz` limit 2;