What is a standard way of profiling Scala method calls?
What I need are hooks around a method, using which I can use to start and stop Timers.
In Java I use aspect programming, aspectJ, to define the methods to be profiled and inject bytecode to achieve the same.
Is there a more natural way in Scala, where I can define a bunch of functions to be called before and after a function without losing any static typing in the process?
Do you want to do this without changing the code that you want to measure timings for? If you don't mind changing the code, then you could do something like this:
def time[R](block: => R): R = {
val t0 = System.nanoTime()
val result = block // call-by-name
val t1 = System.nanoTime()
println("Elapsed time: " + (t1 - t0) + "ns")
result
}
// Now wrap your method calls, for example change this...
val result = 1 to 1000 sum
// ... into this
val result = time { 1 to 1000 sum }
In addition to Jesper's answer, you can automatically wrap method invocations in the REPL:
scala> def time[R](block: => R): R = {
| val t0 = System.nanoTime()
| val result = block
| println("Elapsed time: " + (System.nanoTime - t0) + "ns")
| result
| }
time: [R](block: => R)R
Now - let's wrap anything in this
scala> :wrap time
wrap: no such command. Type :help for help.
OK - we need to be in power mode
scala> :power
** Power User mode enabled - BEEP BOOP SPIZ **
** :phase has been set to 'typer'. **
** scala.tools.nsc._ has been imported **
** global._ and definitions._ also imported **
** Try :help, vals.<tab>, power.<tab> **
Wrap away
scala> :wrap time
Set wrapper to 'time'
scala> BigDecimal("1.456")
Elapsed time: 950874ns
Elapsed time: 870589ns
Elapsed time: 902654ns
Elapsed time: 898372ns
Elapsed time: 1690250ns
res0: scala.math.BigDecimal = 1.456
I have no idea why that printed stuff out 5 times
Update as of 2.12.2:
scala> :pa
// Entering paste mode (ctrl-D to finish)
package wrappers { object wrap { def apply[A](a: => A): A = { println("running...") ; a } }}
// Exiting paste mode, now interpreting.
scala> $intp.setExecutionWrapper("wrappers.wrap")
scala> 42
running...
res2: Int = 42
This what I use:
import System.nanoTime
def profile[R](code: => R, t: Long = nanoTime) = (code, nanoTime - t)
// usage:
val (result, time) = profile {
/* block of code to be profiled*/
}
val (result2, time2) = profile methodToBeProfiled(foo)
There are three benchmarking libraries for Scala that you can avail of.
Since the URLs on the linked site are likely to change, I am pasting the relevant content below.
SPerformance - Performance Testing framework aimed at automagically comparing performance tests and working inside Simple Build Tool.
scala-benchmarking-template - SBT template project for creating Scala (micro-)benchmarks based on Caliper.
Metrics - Capturing JVM- and application-level metrics. So you know what's going on
testing.Benchmark might be useful.
scala> def testMethod {Thread.sleep(100)}
testMethod: Unit
scala> object Test extends testing.Benchmark {
| def run = testMethod
| }
defined module Test
scala> Test.main(Array("5"))
$line16.$read$$iw$$iw$Test$ 100 100 100 100 100
I use a technique that's easy to move around in code blocks. The crux is that the same exact line starts and ends the timer - so it is really a simple copy and paste. The other nice thing is that you get to define what the timing means to you as a string, all in that same line.
Example usage:
Timelog("timer name/description")
//code to time
Timelog("timer name/description")
The code:
object Timelog {
val timers = scala.collection.mutable.Map.empty[String, Long]
//
// Usage: call once to start the timer, and once to stop it, using the same timer name parameter
//
def timer(timerName:String) = {
if (timers contains timerName) {
val output = s"$timerName took ${(System.nanoTime() - timers(timerName)) / 1000 / 1000} milliseconds"
println(output) // or log, or send off to some performance db for analytics
}
else timers(timerName) = System.nanoTime()
}
Pros:
no need to wrap code as a block or manipulate within lines
can easily move the start and end of the timer among code lines when being exploratory
Cons:
less shiny for utterly functional code
obviously this object leaks map entries if you do not "close" timers,
e.g. if your code doesn't get to the second invocation for a given timer start.
ScalaMeter is a nice library to perform benchmarking in Scala
Below is a simple example
import org.scalameter._
def sumSegment(i: Long, j: Long): Long = (i to j) sum
val (a, b) = (1, 1000000000)
val execution_time = measure { sumSegment(a, b) }
If you execute above code snippet in Scala Worksheet you get the running time in milliseconds
execution_time: org.scalameter.Quantity[Double] = 0.260325 ms
The recommended approach to benchmarking Scala code is via sbt-jmh
"Trust no one, bench everything." - sbt plugin for JMH (Java
Microbenchmark Harness)
This approach is taken by many of the major Scala projects, for example,
Scala programming language itself
Dotty (Scala 3)
cats library for functional programming
Metals language server for IDEs
Simple wrapper timer based on System.nanoTime is not a reliable method of benchmarking:
System.nanoTime is as bad as String.intern now: you can use it,
but use it wisely. The latency, granularity, and scalability effects
introduced by timers may and will affect your measurements if done
without proper rigor. This is one of the many reasons why
System.nanoTime should be abstracted from the users by benchmarking
frameworks
Furthermore, considerations such as JIT warmup, garbage collection, system-wide events, etc. might introduce unpredictability into measurements:
Tons of effects need to be mitigated, including warmup, dead code
elimination, forking, etc. Luckily, JMH already takes care of many
things, and has bindings for both Java and Scala.
Based on Travis Brown's answer here is an example of how to setup JMH benchmark for Scala
Add jmh to project/plugins.sbt
addSbtPlugin("pl.project13.scala" % "sbt-jmh" % "0.3.7")
Enable jmh plugin in build.sbt
enablePlugins(JmhPlugin)
Add to src/main/scala/bench/VectorAppendVsListPreppendAndReverse.scala
package bench
import org.openjdk.jmh.annotations._
#State(Scope.Benchmark)
#BenchmarkMode(Array(Mode.AverageTime))
class VectorAppendVsListPreppendAndReverse {
val size = 1_000_000
val input = 1 to size
#Benchmark def vectorAppend: Vector[Int] =
input.foldLeft(Vector.empty[Int])({ case (acc, next) => acc.appended(next)})
#Benchmark def listPrependAndReverse: List[Int] =
input.foldLeft(List.empty[Int])({ case (acc, next) => acc.prepended(next)}).reverse
}
Execute benchmark with
sbt "jmh:run -i 10 -wi 10 -f 2 -t 1 bench.VectorAppendVsListPreppendAndReverse"
The results are
Benchmark Mode Cnt Score Error Units
VectorAppendVsListPreppendAndReverse.listPrependAndReverse avgt 20 0.024 ± 0.001 s/op
VectorAppendVsListPreppendAndReverse.vectorAppend avgt 20 0.130 ± 0.003 s/op
which seems to indicate prepending to a List and then reversing it at the end is order of magnitude faster than keep appending to a Vector.
I took the solution from Jesper and added some aggregation to it on multiple run of the same code
def time[R](block: => R) = {
def print_result(s: String, ns: Long) = {
val formatter = java.text.NumberFormat.getIntegerInstance
println("%-16s".format(s) + formatter.format(ns) + " ns")
}
var t0 = System.nanoTime()
var result = block // call-by-name
var t1 = System.nanoTime()
print_result("First Run", (t1 - t0))
var lst = for (i <- 1 to 10) yield {
t0 = System.nanoTime()
result = block // call-by-name
t1 = System.nanoTime()
print_result("Run #" + i, (t1 - t0))
(t1 - t0).toLong
}
print_result("Max", lst.max)
print_result("Min", lst.min)
print_result("Avg", (lst.sum / lst.length))
}
Suppose you want to time two functions counter_new and counter_old, the following is the usage:
scala> time {counter_new(lst)}
First Run 2,963,261,456 ns
Run #1 1,486,928,576 ns
Run #2 1,321,499,030 ns
Run #3 1,461,277,950 ns
Run #4 1,299,298,316 ns
Run #5 1,459,163,587 ns
Run #6 1,318,305,378 ns
Run #7 1,473,063,405 ns
Run #8 1,482,330,042 ns
Run #9 1,318,320,459 ns
Run #10 1,453,722,468 ns
Max 1,486,928,576 ns
Min 1,299,298,316 ns
Avg 1,407,390,921 ns
scala> time {counter_old(lst)}
First Run 444,795,051 ns
Run #1 1,455,528,106 ns
Run #2 586,305,699 ns
Run #3 2,085,802,554 ns
Run #4 579,028,408 ns
Run #5 582,701,806 ns
Run #6 403,933,518 ns
Run #7 562,429,973 ns
Run #8 572,927,876 ns
Run #9 570,280,691 ns
Run #10 580,869,246 ns
Max 2,085,802,554 ns
Min 403,933,518 ns
Avg 797,980,787 ns
Hopefully this is helpful
I like the simplicity of #wrick's answer, but also wanted:
the profiler handles looping (for consistency and convenience)
more accurate timing (using nanoTime)
time per iteration (not total time of all iterations)
just return ns/iteration - not a tuple
This is achieved here:
def profile[R] (repeat :Int)(code: => R, t: Long = System.nanoTime) = {
(1 to repeat).foreach(i => code)
(System.nanoTime - t)/repeat
}
For even more accuracy, a simple modification allows a JVM Hotspot warmup loop (not timed) for timing small snippets:
def profile[R] (repeat :Int)(code: => R) = {
(1 to 10000).foreach(i => code) // warmup
val start = System.nanoTime
(1 to repeat).foreach(i => code)
(System.nanoTime - start)/repeat
}
You can use System.currentTimeMillis:
def time[R](block: => R): R = {
val t0 = System.currentTimeMillis()
val result = block // call-by-name
val t1 = System.currentTimeMillis()
println("Elapsed time: " + (t1 - t0) + "ms")
result
}
Usage:
time{
//execute somethings here, like methods, or some codes.
}
nanoTime will show you ns, so it will hard to see. So I suggest that you can use currentTimeMillis instead of it.
While standing on the shoulders of giants...
A solid 3rd-party library would be more ideal, but if you need something quick and std-library based, following variant provides:
Repetitions
Last result wins for multiple repetitions
Total time and average time for multiple repetitions
Removes the need for time/instant provider as a param
.
import scala.concurrent.duration._
import scala.language.{postfixOps, implicitConversions}
package object profile {
def profile[R](code: => R): R = profileR(1)(code)
def profileR[R](repeat: Int)(code: => R): R = {
require(repeat > 0, "Profile: at least 1 repetition required")
val start = Deadline.now
val result = (1 until repeat).foldLeft(code) { (_: R, _: Int) => code }
val end = Deadline.now
val elapsed = ((end - start) / repeat)
if (repeat > 1) {
println(s"Elapsed time: $elapsed averaged over $repeat repetitions; Total elapsed time")
val totalElapsed = (end - start)
println(s"Total elapsed time: $totalElapsed")
}
else println(s"Elapsed time: $elapsed")
result
}
}
Also worth noting you can use the Duration.toCoarsest method to convert to the biggest time unit possible, although I am not sure how friendly this is with minor time difference between runs e.g.
Welcome to Scala version 2.11.7 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_60).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import scala.concurrent.duration._
import scala.concurrent.duration._
scala> import scala.language.{postfixOps, implicitConversions}
import scala.language.{postfixOps, implicitConversions}
scala> 1000.millis
res0: scala.concurrent.duration.FiniteDuration = 1000 milliseconds
scala> 1000.millis.toCoarsest
res1: scala.concurrent.duration.Duration = 1 second
scala> 1001.millis.toCoarsest
res2: scala.concurrent.duration.Duration = 1001 milliseconds
scala>
adding on => method with name & seconds
profile[R](block: => R,methodName : String): R = {
val n = System.nanoTime()
val result = block
val n1 = System.nanoTime()
println(s"Elapsed time: ${TimeUnit.MILLISECONDS.convert(n1 - n,TimeUnit.NANOSECONDS)}ms - MethodName: ${methodName}")
result
}
Related
I follow a python course on finance about portfolio theory. I have to create a function with a nested function in it.
My problem is I have a error message of "neg_sharpe_ratio() missing 2 required positional arguments: 'er' and 'cov'" whereas to my mind 'er' and 'cov' are already defined in my function msr below. So I understand how they are missing.
from scipy.optimize import minimize
def msr(riskfree_rate, er, cov):
n= er.shape[0]
init_guess= np.repeat(1/n, n)
bounds=((0.00, 1.0),)*n
weights_sum_to_1 = {
'type' :'eq' , #
'fun' : lambda weights: np.sum(weights) - 1 ##
}
def neg_sharpe_ratio(weights,riskfree_rate, er, cov):
r = erk.portfolio_return(weights, er)
vol = erk.portfolio_vol(weights,cov)
return -(r-riskfree_rate)/vol
results = minimize( neg_sharpe_ratio, init_guess,
args=(cov,), method="SLSQP",
options={'disp': False},
constraints=( weights_sum_to_1),
bounds=bounds
)
return results.x
TypeError: neg_sharpe_ratio() missing 2 required positional arguments: 'er' and 'cov'
The function neg_sharpe_ratio is able to reference any of the variables passed in and made by the function msr without needing those same variables passed into it itself. Therefore you should be able to remove the paramters riskfree_rate, er, and cov from the neq_sharpe_ratio function definition and have it work, as those variables are passed into its parent function, leaving you with:
def neg_sharpe_ratio(weights):
For those who might be interested, I find my mistake..
Indeed, I forgot to define correctly the arguments of my function neg_share_ratio in my function minimize.
Here is the code amended:
from scipy.optimize import minimize
def msr(riskfree_rate, er, cov):
n= er.shape[0]
init_guess= np.repeat(1/n, n)
bounds=((0.00, 1.0),)*n
weights_sum_to_1 = {
'type' :'eq' , #
'fun' : lambda weights: np.sum(weights) - 1 ##
}
def neg_sharpe_ratio(weights,riskfree_rate, er, cov):
r = erk.portfolio_return(weights, er)
vol = erk.portfolio_vol(weights,cov)
return -(r-riskfree_rate)/vol
results = minimize( neg_sharpe_ratio, init_guess,
args=(weights,riskfree_rate,er,cov), method="SLSQP",
options={'disp': False},
constraints=( weights_sum_to_1),
bounds=bounds
)
return results.x code here
I'm new to reinforcement learning, and I would like to process audio signal using this technique. I built a basic step function that I wish to flatten to get my hands on Gym OpenAI and reinforcement learning in general.
To do so, I am using the GoalEnv provided by OpenAI since I know what the target is, the flat signal.
That is the image with input and desired signal :
The step function calls _set_action which performs achieved_signal = convolution(input_signal,low_pass_filter) - offset, low_pass_filter takes a cutoff frequency as input as well.
Cutoff frequency and offset are the parameters that act on the observation to get the output signal.
The designed reward function returns the frame to frame L2-norm between the input signal and the desired signal, to the negative, to penalize a large norm.
Following is the environment I created:
def butter_lowpass(cutoff, nyq_freq, order=4):
normal_cutoff = float(cutoff) / nyq_freq
b, a = signal.butter(order, normal_cutoff, btype='lowpass')
return b, a
def butter_lowpass_filter(data, cutoff_freq, nyq_freq, order=4):
b, a = butter_lowpass(cutoff_freq, nyq_freq, order=order)
y = signal.filtfilt(b, a, data)
return y
class `StepSignal(gym.GoalEnv)`:
def __init__(self, input_signal, sample_rate, desired_signal):
super(StepSignal, self).__init__()
self.initial_signal = input_signal
self.signal = self.initial_signal.copy()
self.sample_rate = sample_rate
self.desired_signal = desired_signal
self.distance_threshold = 10e-1
max_offset = abs(max( max(self.desired_signal) , max(self.signal))
- min( min(self.desired_signal) , min(self.signal)) )
self.action_space = spaces.Box(low=np.array([10e-4,-max_offset]),\
high=np.array([self.sample_rate/2-0.1,max_offset]), dtype=np.float16)
obs = self._get_obs()
self.observation_space = spaces.Dict(dict(
desired_goal=spaces.Box(-np.inf, np.inf, shape=obs['achieved_goal'].shape, dtype='float32'),
achieved_goal=spaces.Box(-np.inf, np.inf, shape=obs['achieved_goal'].shape, dtype='float32'),
observation=spaces.Box(-np.inf, np.inf, shape=obs['observation'].shape, dtype='float32'),
))
def step(self, action):
range = self.action_space.high - self.action_space.low
action = range / 2 * (action + 1)
self._set_action(action)
obs = self._get_obs()
done = False
info = {
'is_success': self._is_success(obs['achieved_goal'], self.desired_signal),
}
reward = -self.compute_reward(obs['achieved_goal'],self.desired_signal)
return obs, reward, done, info
def reset(self):
self.signal = self.initial_signal.copy()
return self._get_obs()
def _set_action(self, actions):
actions = np.clip(actions,a_max=self.action_space.high,a_min=self.action_space.low)
cutoff = actions[0]
offset = actions[1]
print(cutoff, offset)
self.signal = butter_lowpass_filter(self.signal, cutoff, self.sample_rate/2) - offset
def _get_obs(self):
obs = self.signal
achieved_goal = self.signal
return {
'observation': obs.copy(),
'achieved_goal': achieved_goal.copy(),
'desired_goal': self.desired_signal.copy(),
}
def compute_reward(self, goal_achieved, goal_desired):
d = np.linalg.norm(goal_desired-goal_achieved)
return d
def _is_success(self, achieved_goal, desired_goal):
d = self.compute_reward(achieved_goal, desired_goal)
return (d < self.distance_threshold).astype(np.float32)
The environment can then be instantiated into a variable, and flattened through the FlattenDictWrapper as advised here https://openai.com/blog/ingredients-for-robotics-research/ (end of the page).
length = 20
sample_rate = 30 # 30 Hz
in_signal_length = 20*sample_rate # 20sec signal
x = np.linspace(0, length, in_signal_length)
# Desired output
y = 3*np.ones(in_signal_length)
# Step signal
in_signal = 0.5*(np.sign(x-5)+9)
env = gym.make('stepsignal-v0', input_signal=in_signal, sample_rate=sample_rate, desired_signal=y)
env = gym.wrappers.FlattenDictWrapper(env, dict_keys=['observation','desired_goal'])
env.reset()
The agent is a DDPG Agent from keras-rl, since the actions can take any values in the continuous action_space described in the environment.
I wonder why the actor and critic nets need an input with an additional dimension, in input_shape=(1,) + env.observation_space.shape
nb_actions = env.action_space.shape[0]
# Building Actor agent (Policy-net)
actor = Sequential()
actor.add(Flatten(input_shape=(1,) + env.observation_space.shape, name='flatten'))
actor.add(Dense(128))
actor.add(Activation('relu'))
actor.add(Dense(64))
actor.add(Activation('relu'))
actor.add(Dense(nb_actions))
actor.add(Activation('linear'))
actor.summary()
# Building Critic net (Q-net)
action_input = Input(shape=(nb_actions,), name='action_input')
observation_input = Input(shape=(1,) + env.observation_space.shape, name='observation_input')
flattened_observation = Flatten()(observation_input)
x = Concatenate()([action_input, flattened_observation])
x = Dense(128)(x)
x = Activation('relu')(x)
x = Dense(64)(x)
x = Activation('relu')(x)
x = Dense(1)(x)
x = Activation('linear')(x)
critic = Model(inputs=[action_input, observation_input], outputs=x)
critic.summary()
# Building Keras agent
memory = SequentialMemory(limit=2000, window_length=1)
policy = BoltzmannQPolicy()
random_process = OrnsteinUhlenbeckProcess(size=nb_actions, theta=0.6, mu=0, sigma=0.3)
agent = DDPGAgent(nb_actions=nb_actions, actor=actor, critic=critic, critic_action_input=action_input,
memory=memory, nb_steps_warmup_critic=2000, nb_steps_warmup_actor=10000,
random_process=random_process, gamma=.99, target_model_update=1e-3)
agent.compile(Adam(lr=1e-3, clipnorm=1.), metrics=['mae'])
Finally, the agent is trained:
filename = 'mem20k_heaviside_flattening'
hist = agent.fit(env, nb_steps=10, visualize=False, verbose=2, nb_max_episode_steps=5)
with open('./history_dqn_test_'+ filename + '.pickle', 'wb') as handle:
pickle.dump(hist.history, handle, protocol=pickle.HIGHEST_PROTOCOL)
agent.save_weights('h5f_files/dqn_{}_weights.h5f'.format(filename), overwrite=True)
Now here is the catch: the agent seems to always be stuck to the same neighborhood of output values across all episodes for a same instance of my env:
The cumulated reward is negative since I just allowed the agent to get negative rewards. I used it from https://github.com/openai/gym/blob/master/gym/envs/robotics/fetch_env.py which is part of OpenAI code as example.
Across one episode, I should get varying sets of actions converging towards a (cutoff_final, offset_final) that would get my input step signal close to my output flat signal, which is clearly not the case. In addition, I thought, for successive episodes, I should get different actions.
I wonder why the actor and critic nets need an input with an additional dimension, in input_shape=(1,) + env.observation_space.shape
I think the GoalEnv is designed with HER (Hindsight Experience Replay) in mind, since it will use the "sub-spaces" inside the observation_space to learn from sparse reward signals (there is a paper in OpenAI website that explains how HER works). Haven't look at the implementation, but my guess is that there needs to be an additional input since HER also process the "goal" parameter.
Since it seems you are not using HER (works with any off-policy algorithm, including DQN, DDPG, etc), you should handcraft an informative reward function (rewards are not binary, eg, 1 if objective achieved, 0 otherwise) and use the base Env class. The reward should be calculated inside the step method, since rewards in MDP's are functions like r(s, a, s`) you probably will have all the information you need. Hope it helps.
I'm trying to build a power analysis tool in Scala. In particular, I'm trying to create a function which does return the required sample size given some restrictions (statistical power, effect size, the ratio between the two sample sizes, significance level and the type of hypothesis test we would like to run).
I wrote the following code:
import org.apache.commons.math3.analysis.UnivariateFunction
import org.apache.commons.math3.analysis.solvers.BrentSolver
import org.apache.commons.math3.distribution.NormalDistribution
import scala.math._
object normal_sample_size extends App {
val theta1 = 0.0025
val theta2 = theta1 * 1.05
val split = 0.50
val power = 0.80
val significance = 0.05
val alternative = "two sided"
val result = requiredSizeCalculationNormal(effectSizeCalculation(theta1, theta2), split, power, significance, alternative)
println(result)
def effectSizeCalculation(mu1: Double, mu2: Double): Double = {
2 * math.asin(math.sqrt(mu1)) - 2 * math.asin(math.sqrt(mu2))
}
def requiredSizeCalculationNormal(effect_size: Double, split_ratio: Double, power: Double, significance: Double, alternative: String): Unit = {
if (alternative == "two sided") {
val z_score = new NormalDistribution().inverseCumulativeProbability(significance / 2.0)
val func = new UnivariateFunction {def value(n: Double) = (1 - new NormalDistribution().cumulativeProbability(z_score.abs - effect_size * sqrt(n * split_ratio * n * (1 - split_ratio)))) +
new NormalDistribution().cumulativeProbability(z_score - effect_size * sqrt(n * split_ratio * n * (1 - split_ratio)))}
val solver = new BrentSolver()
try {solver.solve(500, func, 0.001, 1000000)} catch {case _:Throwable => None}
} else if (alternative == "less") {
//val z_score = new NormalDistribution().inverseCumulativeProbability(significance)
//new NormalDistribution().cumulativeProbability(z_score - effect_size * sqrt(n * split_ratio * n * (1 - split_ratio)))
} else if (alternative == "greater") {
//val z_score = new NormalDistribution().inverseCumulativeProbability(significance).abs
//1 - new NormalDistribution().cumulativeProbability(z_score - effect_size * sqrt(n * split_ratio * n * (1 - split_ratio)))
} else {
println("Error message: Specify the right alternative. Possible values are 'two sided', 'less', 'greater'")
}
}
}
requiredSizeCalculationNormal() recognises what type of hypothesis testing is dealing with ("two sided", "greater" or "less") and then computes the required sample size. This last point is where I'm struggling! Using Brent's Method (from math3 library) I would like to find the value of n which solve the function.
Going through the code, focusing only on the "two sided" part of my function (I'll complete "greater" and "less" in a second stage), I've tried to create a function named func. Then using the command try {solver.solve(500, func, 0.001, 1000000)} catch {case _:Throwable => None} I'm trying to solve it using max 500 iterations and within the interval (0.001, 1000000). Unfortunately the command doesn't return neither a result nor an error message.
EDIT: I've removed from the code quoted above the //catch {case _:Throwable => None} (None was redundant anyway). That piece of code was actually catching all exceptions and doing nothing with them.
The exceptions I see running the code are:
Exception in thread "main" org.apache.commons.math3.exception.NoBracketingException: function values at endpoints do not have different signs, endpoints: [2, 10,000,000], values: [0.05, 1]
at org.apache.commons.math3.analysis.solvers.BrentSolver.doSolve(BrentSolver.java:118)
at org.apache.commons.math3.analysis.solvers.BaseAbstractUnivariateSolver.solve(BaseAbstractUnivariateSolver.java:190)
at org.apache.commons.math3.analysis.solvers.BaseAbstractUnivariateSolver.solve(BaseAbstractUnivariateSolver.java:195)
at normal_sample_size$.requiredSizeCalculationNormal(normal_sample_size.scala:46)
at normal_sample_size$.delayedEndpoint$normal_sample_size$1(normal_sample_size.scala:31)
at normal_sample_size$delayedInit$body.apply(normal_sample_size.scala:22)
at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:381)
at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
at scala.App$class.main(App.scala:76)
at normal_sample_size$.main(normal_sample_size.scala:22)
at normal_sample_size.main(normal_sample_size.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
Process finished with exit code 1
So the problem seems to be that the function doesn't change sign within the specified interval.
I have the following code:
{-# LANGUAGE DeriveDataTypeable #-}
import Prelude hiding (catch)
import Control.Exception (throwIO, Exception)
import Control.Monad (when)
import Data.Maybe
import Data.Word (Word16)
import Data.Typeable (Typeable)
import System.Environment (getArgs)
data ArgumentParserException = WrongArgumentCount | InvalidPortNumber
deriving (Show, Typeable)
instance Exception ArgumentParserException
data Arguments = Arguments Word16 FilePath String
main = do
args <- return []
when (length args /= 3) (throwIO WrongArgumentCount)
let [portStr, cert, pw] = args
let portInt = readMaybe portStr :: Maybe Integer
when (portInt == Nothing) (throwIO InvalidPortNumber)
let portNum = fromJust portInt
when (portNum < 0 || portNum > 65535) (throwIO InvalidPortNumber)
return $ Arguments (fromInteger portNum) cert pw
-- Newer 'base' has Text.Read.readMaybe but alas, that doesn't come with
-- the latest Haskell platform, so let's not rely on it
readMaybe :: Read a => String -> Maybe a
readMaybe s = case reads s of
[(x, "")] -> Just x
_ -> Nothing
Its behavior differs when compiled with optimizations on and off:
crabgrass:~/tmp/signserv/src% ghc -fforce-recomp Main.hs && ./Main
Main: WrongArgumentCount
crabgrass:~/tmp/signserv/src% ghc -O -fforce-recomp Main.hs && ./Main
Main: Main.hs:20:9-34: Irrefutable pattern failed for pattern [portStr, cert, pw]
Why is this? I am aware that imprecise exceptions can be chosen from arbitrarily; but here we are choosing from one precise and one imprecise exception, so that caveat should not apply.
I would agree with hammar, this looks like a bug. And it seems fixed in HEAD since some time. With an older ghc-7.7.20130312 as well as with today's HEAD ghc-7.7.20130521, the WrongArgumentCount exception is raised and all the other code of main is removed (bully for the optimiser). Still broken in 7.6.3, however.
The behaviour changed with the 7.2 series, I get the expected WrongArgumentCount from 7.0.4, and the (optimised) core makes that clear:
Main.main1 =
\ (s_a11H :: GHC.Prim.State# GHC.Prim.RealWorld) ->
case GHC.List.$wlen
# GHC.Base.String (GHC.Types.[] # GHC.Base.String) 0
of _ {
__DEFAULT ->
case GHC.Prim.raiseIO#
# GHC.Exception.SomeException # () Main.main7 s_a11H
of _ { (# new_s_a11K, _ #) ->
Main.main2 new_s_a11K
};
3 -> Main.main2 s_a11H
}
when the length of the empty list is different from 3, raise WrongArgumentCount, otherwise try to do the rest.
With 7.2 and later, the evaluation of the length is moved behind the parsing of portStr:
Main.main1 =
\ (eta_Xw :: GHC.Prim.State# GHC.Prim.RealWorld) ->
case Main.main7 of _ {
[] -> case Data.Maybe.fromJust1 of wild1_00 { };
: ds_dTy ds1_dTz ->
case ds_dTy of _ { (x_aOz, ds2_dTA) ->
case ds2_dTA of _ {
[] ->
case ds1_dTz of _ {
[] ->
case GHC.List.$wlen
# [GHC.Types.Char] (GHC.Types.[] # [GHC.Types.Char]) 0
of _ {
__DEFAULT ->
case GHC.Prim.raiseIO#
# GHC.Exception.SomeException # () Main.main6 eta_Xw
of wild4_00 {
};
3 ->
where
Main.main7 =
Text.ParserCombinators.ReadP.run
# GHC.Integer.Type.Integer Main.main8 Main.main3
Main.main8 =
GHC.Read.$fReadInteger5
GHC.Read.$fReadInteger_$sconvertInt
Text.ParserCombinators.ReadPrec.minPrec
# GHC.Integer.Type.Integer
(Text.ParserCombinators.ReadP.$fMonadP_$creturn
# GHC.Integer.Type.Integer)
Main.main3 = case lvl_r1YS of wild_00 { }
lvl_r1YS =
Control.Exception.Base.irrefutPatError
# ([GHC.Types.Char], [GHC.Types.Char], [GHC.Types.Char])
"Except.hs:21:9-34|[portStr, cert, pw]"
Since throwIO is supposed to respect ordering of IO actions,
The throwIO variant should be used in preference to throw to raise an exception within the IO monad because it guarantees ordering with respect to other IO operations, whereas throw does not.
that should not happen.
You can force the correct ordering by using a NOINLINE variant of when, or by performing an effectful IO action before throwing, so it seems that when the inliner sees that the when does nothing except possibly throwing, it decides that order doesn't matter.
(Sorry, not a real answer, but try to fit that in a comment ;)
I am interested in using the monotone spline, but I get an error when R tries to use it. I am using R 2.12.0, and the method 'monoH.FC' says that it has been supported since 2.8.0
Reproducible example (same result for more complicated (x,y) relationships)
x<-1:2
y<-1:2
spline(x,y,method="monoH.FC")
Error in spline(x, y, method = "monoH.FC") : invalid interpolation method
What I have tried
?spline returns:
...
Usage:
...
spline(x, y = NULL, n = 3*length(x), method = "fmm",
xmin = min(x), xmax = max(x), xout, ties = mean)
...
Arguments:
method: specifies the type of spline to be used. Possible values are
‘"fmm"’, ‘"natural"’, ‘"periodic"’ and ‘"monoH.FC"’.
...
But the spline function itself indicates that the 'monoH.FC' method is not supported:
...
method <- pmatch(method, c("periodic", "natural", "fmm"))
if (is.na(method))
stop("invalid interpolation method")
...
Question
How can I use method = 'monoH.FC' with spline?
Use splinefun; it supports method=monoH.FC.
The last example in ?spline shows you how to do it.
## An example of monotone interpolation
n <- 20
set.seed(11)
x. <- sort(runif(n)) ; y. <- cumsum(abs(rnorm(n)))
plot(x.,y.)
curve(splinefun(x.,y.)(x), add=TRUE, col=2, n=1001)
curve(splinefun(x.,y., method="mono")(x), add=TRUE, col=3, n=1001)
legend("topleft", paste("splinefun( \"", c("fmm", "monoH.CS"), "\" )", sep=''),
col=2:3, lty=1)