I have been tasked with templating MySQL's my.cnf in an attempt standardize the configuration amongst the slave databases using Puppet. Right now, I'm targeting the innodb settings.
Are there configuration options that can safely be calculated against hardware specifications such as memory, disk and procs?
You need facter.
puppet:/etc/puppet/modules/master/lib/facter$ cat disks.rb
#!/usr/bin/ruby
#require 'facter'
mount = `/bin/mount`
disks=Array.new
mount.split("\n").each_with_index { | disk,i |
unless disk.scan(/ext3|simfs|reiserfs|xfs/).empty?
d=disk.split[2]
disks.push d
disks.push ','
end
}
Facter.add('disks') do
setcode do
disks
end
end
`
and in puppet.pp i use facts $disks
#add disk check to zabbix
exec { "create_host":
command => "/bin/echo $fqdn $ipaddress $disks | do_work",
require => File["/root/ticket"],
subscribe => File["/root/ticket"],
refreshonly => true,
}
see "Adding Custom Facts to Facter" on puppet labs.
I'd be tempted to move the calculations into the erb file, for example the key_buffer_size is recommended to be set to 1/4 of the Systems RAM:
set-variable = key_buffer_size=<%= (memorysize.split(' ')[0].to_i * 1024) / 4 -%>M
there is no reason why you couldn't work on other variables available from Facter (number of processors etc) and come up with your own calculations to set other variables as above.
Remember ERB effectively provides a subset of Ruby so almost anything you can do in Ruby can be done in ERB.
puppet have the erb template, erb template can use the facter value ,like hostname or memor. and you can write you self facter shell script.
Related
I am using bazel for building bare metal programs. I would like to run the unit tests generated by bazel on qemu.
qemu-system-* -some_args -kernel bazel-bin/whatever/generated.elf
I've tried to run those by creating my own rule in a '.bzl'-file, but it seems that outputs are mandatory on all the rule actions. Note, that I need to invoke different qemu commands with different arguments depending on the target architecture. I would like to pass those to the rule.
Is there a way to invoke a shell command without any outputs?
If needed, this is what I have so far (yet I'm not sure which parts are correct as bazel stops in the analysis phase):
# run_tests.bzl
===============
def _impl(ctx):
qemu = ctx.attr.qemu
machine = ctx.attr.machine
cpu = ctx.attr.cpu
target = ctx.attr.target
# The command may only access files declared in inputs.
ctx.actions.run_shell(
arguments = [qemu, machine, cpu, target],
command="$1 -M $2 -cpu $3 -nographic -monitor null -serial null -semihosting -kernel $4")
run_tests = rule(
implementation=_impl,
attrs = {"qemu" : attr.string(),
"machine" : attr.string(),
"cpu" : attr.string(),
"target" : attr.string(),},
executable = True
)
And my BUILD-File:
# BUILD
=======
load("//make:run_tests.bzl", "run_tests")
run_tests(
name = "portos",
qemu = "qemu-system-arm",
machine = "realview-pbx-a9",
cpu = "cortex-a9",
target = ":test_portos.elf"
)
cc_binary(
name = "test_portos.elf",
srcs = glob(["*.cc"]),
deps = ["//src:portos",
"#unity//:unity"],
copts = ["-Isrc",
"-Iexternal/unity/src",
"-Iexternal/unity/extras/fixture/src"]
)
You are almost there: yes, you need outputs, otherwise bazel has nothing to do. For the rule output, you probably want the test logs or tests results.
Skylark has support for writing test rules. Basically instead of setting executable = True, you would set test = True, and then your rule would create an executable that is the test, then you set ctx.outputs.executable to that executable. Then you can use the bazel test command with your rule.
See:
docs: https://docs.bazel.build/versions/master/skylark/rules.html#test-rules
example: https://github.com/bazelbuild/examples/tree/master/rules/test_rule
rule.test: https://docs.bazel.build/versions/master/skylark/lib/globals.html#rule.test
I am creating application in Ruby on Rails which is having many engines(for modularity).
I want different databases for each engine. How to configure this?
Database - MYSQL
There is a good explanation by the link http://www.blrice.net/blog/2016/04/09/one-rails-app-with-many-databases/
General approach is to take a look at the framework sources and decide can it be reused.
Let's take a look at activerecord/lib/active_record/railties/databases.rake (v5.0.7) first. For example on how db:create implemented.
We will see ActiveRecord::Tasks::DatabaseTasks.create_current.
Let's open ActiveRecord::Tasks::DatabaseTasks and take a look at
# The possible config values are:
#
# * +env+: current environment (like Rails.env).
# * +database_configuration+: configuration of your databases (as in +config/database.yml+).
# * +db_dir+: your +db+ directory.
# * +fixtures_path+: a path to fixtures directory.
# * +migrations_paths+: a list of paths to directories with migrations.
# * +seed_loader+: an object which will load seeds, it needs to respond to the +load_seed+ method.
# * +root+: a path to the root of the application.
#
# Example usage of DatabaseTasks outside Rails could look as such:
#
# include ActiveRecord::Tasks
# DatabaseTasks.database_configuration = YAML.load_file('my_database_config.yml')
# DatabaseTasks.db_dir = 'db'
# # other settings...
This way we are getting to following solution:
namespace :your_engine do
namespace :db do
task :load_config do
ActiveRecord::Tasks::DatabaseTasks.database_configuration = YAML.load_file("config/database_your_engine.yml")
ActiveRecord::Tasks::DatabaseTasks.db_dir = "db_your_engine"
ActiveRecord::Tasks::DatabaseTasks.migrations_paths = [ "components/your_engine/db/migrate" ]
ActiveRecord::Base.configurations = ActiveRecord::Tasks::DatabaseTasks.database_configuration
ActiveRecord::Migrator.migrations_paths = ActiveRecord::Tasks::DatabaseTasks.migrations_paths
# You can observe following values to see how settings applied.
# puts ActiveRecord::Base.configurations
# puts ActiveRecord::Migrator.migrations_paths
# puts ActiveRecord::Tasks::DatabaseTasks.database_configuration
# puts ActiveRecord::Tasks::DatabaseTasks.migrations_paths
end
desc "Create Your DB"
task create: :load_config do
ActiveRecord::Tasks::DatabaseTasks.create_current
end
end
end
The same approach for drop/migrate and other needed tasks.
It is good general rule - know you stack at least one level lower than your work with. Sometimes reading underlying sources much more helpful than direct answer.
I will update this answer while going forward with my solution...
I wrote a code like this (I use MySQL, PDO, InnoDB, Laravel4, localhost & MAC) :
$all_queue = Queue1::all()->toArray(); //count about 10000
ob_end_clean();
foreach($all_queue as $key=>$value) {
$priceCreate=array(...);
Price::create($priceCreate);
Queue1::where('id',$value['id'])->delete();
}
This worked for me (65mg ram usage), but when it was working, other parts of my program(such as other tables) didn't work. I can't open my database on mysql even. My program and my sql wait and when process is completed ,they work.
I don't know what am i supposed to do.
I think this is not for laravel and this is for my php or mysql configuration.
this is my php.ini and mysql config
I assume
$all_foreach($all_queue as $key=>$value) {
Is
foreach($all_queue as $key=>$value) {
And that you have no errors (you have set debug true in your app config).
Try to set no time limit for your script.
In your php.ini
max_execution_time = 3600 ;this is one hour, set to 0 to no limit
Or in code
set_time_limit(0)
And if it's a memory problem try to free memory and unset unused vars. I'ts a good practice in long scripts to free space.
...
}//end foreach loop
unset($all_queue); //no longer needed, so unset it to free memory
I'd like to know, in a running jruby script, which java options are set. Is there a way to do this?
My problem is this: There are some scripts that I know require much more memory than others, and I would like to add a constraint in the code so that execution will stop early with proper warnings, rather than running out of memory at some unspecified time in the future.
Perhaps something I could stick in a BEGIN{}, like:
if is_set_joption?('-J-Xmx') then
if get_joption('-J-Xmx').match(/\d+/)[0].to_i < 1000 then
puts "You're gonna run out of memory...";
abort();
end
else
puts "I recommend you start with -J-Xmx1000m.";
abort();
end
(... where is_set_joption? and get_joption are made up methods.)
Running jruby 1.7.8.
It'll be in your ENV if you've set JAVA_OPTS in your environment. You could get it from that.. You've already initiated the JVM at this point though, so you'll want to set that elsewhere, like in your command line when you exec jruby with -D
One can do:
require 'java';
java_import 'java.lang.Runtime';
mxm = Runtime.getRuntime.maxMemory.to_i;
if mxm < (512 * 1024 * 1024) then
raise "You're gonna need a bigger boat.";
end
I'm doing my first steps in Puppet and ran into a problem. I've installed PHP on a Linux server and I want to do some slightly changes to php.ini file. I don't want to overwrite the whole ini file with one from repository, just change/create one simple config value.
I want to ensure, that the property upload_max_filesize in php.ini has the value of 10M.
How can I achieve this?
My preferred option would be to leave php.ini alone, and have puppet create a file in php's conf.d directory to override the values you want to change.
The less changes you make to php.ini, the easier it is to see what's going on when you need to merge your changes with the package providers changes when you upgrade php.ini in future.
file {'/etc/php5/conf.d/upload_limits.conf':
ensure => present,
owner => root, group => root, mode => 444,
content => "post_max_size = 10M \nupload_max_filesize = 10M \n",
}
There's basically 3 options:
Use augeas support in puppet (you'll need the ruby augeas libraries installed) like:
augeas { "php.ini":
notify => Service[httpd],
require => Package[php],
context => "/files/etc/php.ini/PHP",
changes => [
"set post_max_size 10M",
"set upload_max_filesize 10M",
];
}
You can use "augtool ls /files/etc/php.ini" to see the sections to understand how augeas is parsing the file and use that to work out the paths you need.
You can use an exec. Something like:
define set_php_var($value) {
exec { "sed -i 's/^;*[[:space:]]*$name[[:space:]]*=.*$/$name = $value/g' /etc/php.ini":
unless => "grep -xqe '$name[[:space:]]*=[[:space:]]*$value' -- /etc/php.ini",
path => "/bin:/usr/bin",
require => Package[php],
notify => Service[httpd];
}
}
set_php_var {
"post_max_size": value => '10M';
"upload_max_filesize": value => '10M';
}
Unfortunately, this solution doesn't understand the sections in php.ini, so adding a variable that's not already there would require extra effort. This will do the wrong thing if a variable appears in more than one section (but examples I'm looking at appear to have all unique variable names). This should work for a variable that's present but commented-out with a semi-colon.
Copy the original php.ini file into your puppet repository and use file with source => 'puppet:///...' or content => template(...) to replace the file entirely, as you indicated you would prefer not to do.
You could also use the file_line resource found in the stdlib module.
file_line{ 'php_upload_max_filesize':
path => '/path/to/php.ini',
line => "upload_max_filesize = 10M",
}
Since this will append the line to the file if one exactly matching it does not exist, and since the last instance of a config value takes precedence over those earlier in the file it will work. This is how I do it when i only have a couple things to change.
An alternative approach, if you're using Apache as your web server, is to set the php variable in your Apache virtualhost file (which will probably be somewhere in your Puppet manifests directory).
For example:
<VirtualHost *:80>
ServerName app.dev
DocumentRoot /srv/app/public
## etc...
php_value upload_max_filesize 10M
</VirtualHost>
This doesn't actually change php.ini, but - depending on your set-up - may be a simple way of achieving the same effect.