I have set direct memory to 44GB
export DRILL_MAX_DIRECT_MEMORY=${DRILL_MAX_DIRECT_MEMORY:-"44G"},
but when I see usage of memory in the Drill dashboard, it's using only 3 or 4GB max. Any idea why ?
I am using ver. 1.14
Also I see that correct memory is assigned to each drillbit when I run
select * from sys.memory
Cheers
This could be a configuration issue:
Is DRILL_MAX_DIRECT_MEMORY env variable already set in your environment? You could use Linux /proc/<pid>/environ to access the Drillbit process environment variables
Make sure that all your Drillbits are configured properly; it is possible that the foreman process didn't get configured properly
Related
I have installed a SpringMVC Web application with JPA and a Mysql Database.
The application is displaying statistics from the database (with a lot of selects)
It works quite fast on Linux(mysql 5.5.54), but it is very slow on Windows 10 (mysql 5.6.38).
Do you know what could cause such a behaviour on Windows?
Or could you give me hints or tell me where to search?
[UPDATE]
Linux : Intel® Core™ i7-4510U CPU # 2.00GHz × 4 / 8GoRAM
Windows : Intel Xeon CPU E31220 3.1Ghz 4GoRAM
I know that the windows machine is not as "powerful" than the linux one. I wonder if, by increasing the memory, that could be enough. Or does Mysql needs a lot of CPU too.
My list would be:
Check configs are identical - not just the settings in my.ini - values not set here are set at compile time and the 2 instances have definitely been compiled seperately! You'll need to capture and compare the output of SHOW VARIABLES
Check file deployment is similar - whether innodb is configured to use one file per table, whether the files are distributed across multiple disks
Check adequate memory available for caching on MSWindows
disable anti-virus
Make sure MSWindows is configured as a server (prioritize background tasks)
Windows sucks, deal with it :)
I can't seem to find a way to start a shell using all the servers set up in conf/servers
Only found it possible to submit to cluster jobs using /bin/snappy-job.sh where I specify the lead location, but would like to try real time shell to perform some tests using the whole cluster
Thank you,
Saif
Please see this link. It tells how to start a spark-shell and connect it to snappy store.
http://snappydatainc.github.io/snappydata/connectingToCluster/#using-the-spark-shell-and-spark-submit
Essentially you need to provide the locator property and this locator is the same which you have used to start the snappy cluster.
$ bin/spark-shell --master local[*] --conf snappydata.store.locators=locatorhost:port --conf spark.ui.port=4041
Note that with the above a different compute cluster is created to run your program. The snappy cluster is not used for computation when you run your code from this shell. The required table definition and data is fetched in efficient fashion from the snappy store.
In future we might make this shell connect to the snappy cluster in such a way that it uses the snappy cluster itself as its compute cluster.
When I run Couchbase server in a Docker container on GCE, using the ncolomer/couchbase image, I'm getting this error:
The maximum number of open files for the couchbase user is set too low.
It must be at least 10240. Normally this can be increased by adding
the following lines to /etc/security/limits.conf:
couchbase soft nofile <value>
couchbase hard nofile <value>
Where <value> is greater than 10240.
The docs in ncolomer/couchbase are recommending to update /etc/init/docker.conf and add limit nofile 262144, but I'm not sure that's even available when using Docker under GCE.
I see a few options:
In the Dockerfile, run a script to modify /etc/security/limits.conf as suggested by the couchbase error.
Call ulimit -n 64000 in the Dockerfile
Any suggestions?
The problem with ulimit is that the limits are bounded by the limits of the docker host process.
This is related to the known #4717 (and to a lesser extent #1916) Docker issue.
As I understand, the two options you mentioned should not work since it will only set ulimit on the child process (i.e. the container). From there, I see no choice but set the correct ulimit on the host before trying to increase it on your container.
The documented procedure should work fine, until you have the possibility to apply it.
I don't know the GCE platform very well, but if you have a root access to your instance, you should just apply the changes to the /etc/init/docker.conf file, restart the Docker service and fire up the Couchbase container.
I'm trying to set up MySQL running in a virtual server (CentOS 6), but as disk intensive stuff isn't great on a VM, I intend to store the database on the host (KVM on CentOS 6) server, and connect to it via Samba or NFS, or the like.
The trouble is that MySQL doesn't seem to like the /var/lib/mysql directory being mounted from a network drive, and I'm getting various different errors in the various configurations I've tried.
My end game is to have the DB server on a VM that can be easily moved between hosts, and the data on a redundant (probably clustered) server. In the mean time, the storage area I'm using on the host server is mirrored using DRBD.
Has anyone done something similar, and can suggest a config that works, or an altogether better way of doing it?
Using a file level protocol is a really bad idea. They are designed to do a very different job.
For block device level protocols there's 2 choices (DRBD doesn't apply here). AoE or iSCSI. IIRC, AoE is tightly coupled to physical network interfaces which may cause some complications in your setup - hence I'd recommend having a long hard look at iSCSI.
I am installing MySQL and when it reaches the configuration step it is asking me to select from on of the following server configuration types:
Development Machine
Server Machine
Dedicated Machine
I am new to mysql, the reason I am installing this is because I am making an application which will store data on a database using MySQL. Now, can someone explain what does these server configurable type mean and which one should I be selecting?
This is only about setting up the initial my.cnf file. You can always change this later.
Generally server configurations allocate a lot more memory to MySQL than you would on your development machine.
This is basically trying to pick the default memory use setting and configuration (over simplification, but good enough for now). If its on a shared system or you have no idea of the physical machine, start with development.
You can always edit the my.cnf to better optimize your setup