First, excuse me for not providing a minimal working example, it is that I just can't think of one, really. I'll just give some pieces of code and ask my question "in principle".
I'm doing thermophysical properties calculation with a real gas model (Peng-Robinson) and here I am having problems when translating a model, where I use pressure p and specific enthalpy h as inputs to calculate all other properties. When it comes to calculating the temperature T, it is linked to the enthalpy h via an equation called departure function, which is itself a function of T. In Modelica it looks like this:
Dh_real = R_m*T*(Z - 1) + (T*dadT - a)/(sqrt(8)*b)*log((Z + (1 + sqrt(2))*B)/(Z + (1 - sqrt(2))*B));
Here a, dadT and Z are also temperature-dependent scalars and partly calculated using matrix operations (dadT) or polynomial-root-calculation (Z) in functions, b and B are parameters.
Calculating the enthalpy from an input temperature (in another model) is straightforward and working fine, the solver can solve the departure function analytically. The other direction has to be solved numerically and this is, I think, why Dymola gives me this error, when translating.
Cannot find differentiation function:
DadT_Unique2([some parameters and T])
with respect to time
Failed to differentiate the equation
dadT = DadT_Unique2([some parameters and T]);
in order to reduce the DAE index.
Failed to reduce the DAE index.
Now DadT is a function within the model, where I use some simple matrix operations to calculate dadT from some parameters and the temperature T. Obviously, Dymola is in need of the derivative of some internal _Unique2-function.
I couldn't find anything in the specification nor in the web about this. Can I provide a derivative of the functions somehow? I tried the smoothOrder-annotation, but without effect. How can I deal with this?
This is not a full answer, but a list of interesting links that you should read:
Michael Tiller on annotation(derivative=dxyz) and other annotations:
http://book.xogeny.com/behavior/functions/func_annos/#derivative
Claytex on numerical Jacobians and flag Hidden.PrintFailureToDifferentiate:
http://www.claytex.com/blog/how-can-i-make-my-models-run-faster/
Two related questions here on StackOverflow:
Dymola solving stationary equation systems for Media-Model
Two-Phase Modelica Media example
Some related Modelica conference papers:
https://modelica.org/events/Conference2005/online_proceedings/Session1/Session1c2.pdf
http://dx.doi.org/10.3384/ecp15118647
http://dx.doi.org/10.3384/ecp15118653
Cubic equation of state, generalized form (table 4.2)
https://books.google.de/books?id=_Op6DQAAQBAJ&pg=PA187
Solving cubic equations of state:
http://dx.doi.org/10.1002/aic.690480421
https://books.google.com/books?id=dd410GGw8wUC&pg=PA48
https://books.google.com/books?id=1rOA5I6kQ7gC&pg=PA620 (Appendix C)
Rewriting partial derivatives:
https://scholar.google.com/scholar?cluster=3379879976574799663
Related
i have been following cs231n lectures of Stanford and trying to complete assignments on my own and sharing these solutions both on github and my blog. But i'm having a hard time on understanding how to modelize backpropagation. I mean i can code modular forward and backward passes but what bothers me is that if i have the model below : Two Layered Neural Network
Lets assume that our loss function here is a softmax loss function. In my modular softmax_loss() function i am calculating loss and gradient with respect to scores (dSoft = dL/dY). After that, when i'am following backwards lets say for b2, db2 would be equal to dSoft*1 or dW2 would be equal to dSoft*dX2(outputs of relu gate). What's the chain rule here ? Why isnt dSoft equal to 1 ? Because dL/dL would be 1 ?
The softmax function is outputs a number given an input x.
What dSoft means is that you're computing the derivative of the function softmax(x) with respect to the input x. Then to calculate the derivative with respect to W of the last layer you use the chain rule i.e. dL/dW = dsoftmax/dx * dx/dW. Note that x = W*x_prev + b where x_prev is the input to the last node. Therefore dx/dW is just x and dx/db is just 1, which means that dL/dW or simply dW is dsoftmax/dx * x_prev and dL/db or simply db is dsoftmax/dx * 1. Note that here dsoftmax/dx is dSoft we defined earlier.
So, I have a vector that corresponds to a given feature (same dimensionality). Is there a package in Julia that would provide a mathematical function that fits these data points, in relation to the original feature? In other words, I have x and y (both vectors) and need to find a decent mapping between the two, even if it's a highly complex one. The output of this process should be a symbolic formula that connects x and y, e.g. (:x)^3 + log(:x) - 4.2454. It's fine if it's just a polynomial approximation.
I imagine this is a walk in the park if you employ Genetic Programming, but I'd rather opt for a simpler (and faster) approach, if it's available. Thanks
Turns out the Polynomials.jl package includes the function polyfit which does Lagrange interpolation. A usage example would go:
using Polynomials # install with Pkg.add("Polynomials")
x = [1,2,3] # demo x
y = [10,12,4] # demo y
polyfit(x,y)
The last line returns:
Poly(-2.0 + 17.0x - 5.0x^2)`
which evaluates to the correct values.
The polyfit function accepts a maximal degree for the output polynomial, but defaults to using the length of the input vectors x and y minus 1. This is the same degree as the polynomial from the Lagrange formula, and since polynomials of such degree agree on the inputs only if they are identical (this is a basic theorem) - it can be certain this is the same Lagrange polynomial and in fact the only one of such a degree to have this property.
Thanks to the developers of Polynomial.jl for leaving me just to google my way to an Answer.
Take a look to MARS regression. Multi adaptive regression splines.
I am attempting to fit a circle to some data. This requires numerically solving a set of three non-linear simultaneous equations (see the Full Least Squares Method of this document).
To me it seems that the NEWTON function provided by IDL is fit for solving this problem. NEWTON requires the name of a function that will compute the values of the equation system for particular values of the independent variables:
FUNCTION newtfunction,X
RETURN, [Some function of X, Some other function of X]
END
While this works fine, it requires that all parameters of the equation system (in this case the set of data points) is hard coded in the newtfunction. This is fine if there is only one data set to solve for, however I have many thousands of data sets, and defining a new function for each by hand is not an option.
Is there a way around this? Is it possible to define functions programmatically in IDL, or even just pass in the data set in some other manner?
I am not an expert on this matter, but if I were to solve this problem I would do the following. Instead of solving a system of 3 non-linear equations to find the three unknowns (i.e. xc, yc and r), I would use an optimization routine to converge to a solution by starting with an initial guess. For this steepest descent, conjugate gradient, or any other multivariate optimization method can be used.
I just quickly derived the least square equation for your problem as (please check before use):
F = (sum_{i=1}^{N} (xc^2 - 2 xi xc + xi^2 + yc^2 - 2 yi yc + yi^2 - r^2)^2)
Calculating the gradient for this function is fairly easy, since it is just a summation, and therefore writing a steepest descent code would be trivial, to calculate xc, yc and r.
I hope it helps.
It's usual to use a COMMON block in these types of functions to pass in other parameters, cached values, etc. that are not part of the calling signature of the numeric routine.
I would like to calculate an integral, which is determined by two functions: I(T) = ∫0T i( f(t), g(t)) dt where f and g solves ordinary differential equations and i is known.
The obvious approach would be to derive a differential equation for I and the solve it alongside f and g (which can be done, but is numerically expensive in my case). In my case, however, f solves an equation with an initial condition f(0) and g and equation with a final condition g(T).
My best guess at the moment would be to solve f and g on a grid using a standard ODE solver and then use a standard method for numerical integration with equally spaced t-coordinates or some kind of quadrature rule (basically anything described by Numerical Recipes).
Does anyone have a better solution? That is, a method that takes the specific type of ode solver and its accuracy into account.
Many advanced ODE solvers come with a feature called "dense output". The ODE solver gives you not only the values of f and g on a grid (as specified beforehand), but allows you to use its result to find the values at any time. Combining this with an adaptive quadrature rule should give you an answer to whatever precision you need.
I have a function f(x) = 1/(x + a+ b*I*sign(x)) and I want to calculate the
integral of
dx dy dz f(x) f(y) f(z) f(x+y+z) f(x-y - z)
over the entire R^3 (b>0 and a,- b are of order unity). This is just a representative example -- in practice I have n<7 variables and 2n-1 instances of f(), n of them involving the n integration variables and n-1 of them involving some linear combintation of the integration variables. At this stage I'm only interested in a rough estimate with relative error of 1e-3 or so.
I have tried the following libraries :
Steven Johnson's cubature code: the hcubature algorithm works but is abysmally slow, taking hundreds of millions of integrand evaluations for even n=2.
HintLib: I tried adaptive integration with a Genz-Malik rule, the cubature routines, VEGAS and MISER with the Mersenne twister RNG. For n=3 only the first seems to be somewhat viable option but it again takes hundreds of millions of integrand evaluations for n=3 and relerr = 1e-2, which is not encouraging.
For the region of integration I have tried both approaches: Integrating over [-200, 200]^n (i.e. a region so large that it essentially captures most of the integral) and the substitution x = sinh(t) which seems to be a standard trick.
I do not have much experience with numerical analysis but presumably the difficulty lies in the discontinuities from the sign() term. For n=2 and f(x)f(y)f(x-y) there are discontinuities along x=0, y=0, x=y. These create a very sharp peak around the origin (with a different sign in the various quadrants) and sort of 'ridges' at x=0,y=0,x=y along which the integrand is large in absolute value and changes sign as you cross them. So at least I know which regions are important. I was thinking that maybe I could do Monte Carlo but somehow "tell" the algorithm in advance where to focus. But I'm not quite sure how to do that.
I would be very grateful if you had any advice on how to evaluate the integral with a reasonable amount of computing power or how to make my Monte Carlo "idea" work. I've been stuck on this for a while so any input would be welcome. Thanks in advance.
One thing you can do is to use a guiding function for your Monte Carlo integration: given an integral (am writing it in 1D for simplicity) of ∫ f(x) dx, write it as ∫ f(x)/g(x) g(x) dx, and use g(x) as a distribution from which you sample x.
Since g(x) is arbitrary, construct it such that (1) it has peaks where you expect them to be in f(x), and (2) such that you can sample x from g(x) (e.g., a gaussian, or 1/(1+x^2)).
Alternatively, you can use a Metropolis-type Markov chain MC. It will find the relevant regions of the integrand (almost) by itself.
Here are a couple of trivial examples.