After updating Openturns from 1.15 to 1.16rc1 I have the following issue with building the meta-model of the field function:
to reduce the computational burden:
ot.ResourceMap.SetAsUnsignedInteger("FittingTest-KolmogorovSamplingSize", 1)
algo = ot.FunctionalChaosAlgorithm(sample_X, outputSampleChaos)
algo.run()
metaModel = ot.PointToFieldConnection(postProcessing, algo.getResult().getMetaModel())
The "FittingTest-KolmogorovSamplingSize" was removed from OpenTurns 1.16rc1 and when I try to replace the fitting test with:
ot.ResourceMap.SetAsUnsignedInteger("FittingTest-LillieforsMaximumSamplingSize", 10)
Or with
ot.ResourceMap.SetAsUnsignedInteger("FittingTest-LillieforsMinimumSamplingSize", 1)
The code is freezing. Is there any solution for this?
The proposed solution is simply to use another distribution to model your data. You could have used any other multivariate continuous distribution of proper dimension. IMO it is not a valid answer as the distribution has no link to your data.
After inspection, it appears that the problem has nothing to do with Lilliefors's test. In OT 1.15 we were using this test (under the wrong name of Kolmogorov) to select automatically a distribution suited to the input sample, but we switched to a more sophisticated selection algorithm (see MetaModelAlgorithm::BuildDistribution). It is based on a first pass using the raw Kolomgorov test (thus ignoring the fact that parameters have been estimated) then an information-based criterion is used to select the most relevant model (AIC, AICC, BIC depending on the value of the "MetaModelAlgorithm-ModelSelectionCriterion" key in ResourceMap. The problem is caused by the TrapezoidalFactory class during the Kolmogorov phase. I will provide a fix ASAP in OpenTURNS master. In the mean time, I have adapted the proposed solution to something more adapted to your data:
degree = 6
dimension_xi_X = 3
dimension_xi_Y = 450
enumerateFunction = ot.HyperbolicAnisotropicEnumerateFunction(dimension_xi_X, 0.8)
basis = ot.OrthogonalProductPolynomialFactory(
[ot.StandardDistributionPolynomialFactory(ot.HistogramFactory().build(sample_X[:,i])) for i in range(dimension_xi_X)], enumerateFunction)
basisSize = enumerateFunction.getStrataCumulatedCardinal(degree)
#basis = ot.OrthogonalProductPolynomialFactory(
# [ot.HermiteFactory()] * dimension_xi_X, enumerateFunction)
#basisSize = 450#enumerateFunction.getStrataCumulatedCardinal(degree)
adaptive = ot.FixedStrategy(basis, basisSize)
projection = ot.LeastSquaresStrategy(
ot.LeastSquaresMetaModelSelectionFactory(ot.LARS(), ot.CorrectedLeaveOneOut()))
ot.ResourceMap.SetAsScalar("LeastSquaresMetaModelSelection-ErrorThreshold", 1.0e-7)
algo_chaos = ot.FunctionalChaosAlgorithm(sample_X,
outputSampleChaos,basis.getMeasure(), adaptive, projection)
algo_chaos.run()
result_chaos = algo_chaos.getResult()
meta_model = result_chaos.getMetaModel()
metaModel = ot.PointToFieldConnection(postProcessing,
algo_chaos.getResult().getMetaModel())
I also implemented a quick and dirty estimator of the L2-error:
# Meta_model validation
iMax = 5
# Input values
sample_X_validation = ot.Sample(np.array(month_1_parameters_MSE.iloc[:iMax,0:3]))
print("sample size=", sample_X_validation.getSize())
# sample_X = ot.Sample(month_1_parameters_MSE[['Rseries','Rsh','Isc']])
# output values
#month_1_simulated.iloc[0:1].transpose()
Field = ot.Field(mesh,np.array(month_1_simulated.iloc[0:1]).transpose())
sample_Y_validation = ot.ProcessSample(1,Field)
for k in range(1,iMax):
sample_Y_validation.add( np.array(month_1_simulated.iloc[k:k+1]).transpose() )
# In[18]:
graph = sample_Y_validation.drawMarginal(0)
graph.setColors(['red'])
drawables = graph.getDrawables()
graph2 = metaModel(sample_X_validation).drawMarginal(0)
graph2.setColors(['blue'])
drawables = graph2.getDrawables()
graph.add(graph2)
graph.setTitle('Model/Metamodel Validation')
graph.setXTitle(r'$t$')
graph.setYTitle(r'$z$')
drawables = graph.getDrawables()
L2_error = 0.0
for i in range(iMax):
L2_error = (drawables[i].getData()[:,1]-drawables[iMax+i].getData()[:,1]).computeRawMoment(2)[0]
print("L2_error=", L2_error)
You get an error of 79.488 with the previous answer and 1.3994 with the new proposal. Here is a graphical comparison.
Comparison between test data & previous answer
Comparison between test data & new proposal
The solution is to use:
degree = 1
dimension_xi_X = 3
dimension_xi_Y = 450
enumerateFunction = ot.LinearEnumerateFunction(dimension_xi_X)
basis = ot.OrthogonalProductPolynomialFactory(
[ot.HermiteFactory()] * dimension_xi_X, enumerateFunction)
basisSize =450 #enumerateFunction.getStrataCumulatedCardinal(degree)
adaptive = ot.FixedStrategy(basis, basisSize)
projection = ot.LeastSquaresStrategy(
ot.LeastSquaresMetaModelSelectionFactory(ot.LARS(), ot.CorrectedLeaveOneOut()))
ot.ResourceMap.SetAsScalar("LeastSquaresMetaModelSelection-ErrorThreshold", 1.0e-7)
algo_chaos = ot.FunctionalChaosAlgorithm(sample_X,
outputSampleChaos,basis.getMeasure(), adaptive, projection)
algo_chaos.run()
result_chaos = algo_chaos.getResult()
meta_model = result_chaos.getMetaModel()
metaModel1 = ot.PointToFieldConnection(postProcessing,
algo_chaos.getResult().getMetaModel())
I have a code that starts as a small amount of variables and makes more elements using those initial variables.
function new( x, y, width, height )
local object = {}
--border
object.border = { x = x, y = y, width = width, height = height }
--body
object.body = { x = x+1, y = y+1, width = width-2, height = height-2 }
--font
object.font = {}
object.font.size = (object.body.height+2)-(math.floor((object.body.height+2)/4)+1)
object.font.height = love.graphics.setNewFont( object.font.size ):getHeight()
--padding
object.padding = {}
object.padding.height = math.floor(object.border.height*(2/29))
object.padding.width = object.padding.height*3
--text
object.text = { input = '' }
object.text.centerHeight = math.ceil(object.body.y+((object.body.height-object.font.height)/2))
object.text.left = object.body.x+object.padding.width+object.padding.height
--backspacing
object.backspace = {key = false, rate = 3, time = 0, pausetime = 20, pause = true}
--config
object.config = { active = true, devmode = false, debug = false, id = gui.id(), type = 'textbox' }
gui.add(object)
return object.config.id
end
and when I modify something in the middle part, the whole thing becomes a mess because starting from the one i changed until the bottom ones value doesn't agree with each other
local x = gui.get(2)
x.body.height = 50
I'm looking if there's a way for these variables to be redefined, starting from them until the bottom, without: (a) making functions for each of the variables. and (b) editing the required parameters in the function.
and If there's none, are the an alternate way to do this efficiently?
EDIT:
the structure of the variables is as follow:
border->body->padding->font
what i needed is a way i can define any of them so that the one that follows also changes like:
object.body.x = 15
and it would collapse from that redefined variable until the bottom:
body->padding->font
i could just redefine them from the edited variable until the bottom like this:
--not the actual code, just an example of variables dependent on the variable above
object.body.x = 15
object.padding.width = object.body.x+1
object.font.size = object.padding.width+1
but that means I have to do the same when redefining the padding until the font which is extremely inefficient especially when I extended more elements.
example:
--padding->font
object.padding.width = 5
object.font.size = object.padding.width+1
I was bored and saw this question (again) along with a duplicate.
I started writing some code for fun, leading to this:
local function getNeededVars(tab,func)
local needed,this = {}
this = setmetatable({},{
__index = function(s,k)
-- See if the requested variable exists.
-- If it doesn't, we obviously complain.
-- If it does, we log it and return the value.
local var = tab.vars[k]
if not var then
error("Eh, "..k.." isn't registered (yet?)",5)
end needed[k] = true return tab.vals[k]
end;
}) func(this) return needed
end
local function updateStuff(self,key,done)
for k,v in pairs(self.levars) do
if v.needed and v.needed[key] then
if not done[v] then done[v] = true
self.vals[v.name] = v.func(self)
updateStuff(self,v.name,done)
end
end
end
end
local function createSubTable(self,key,tab)
return setmetatable({},{
__newindex = function(s,k,v)
tab[k] = v updateStuff(self,key,{})
end; __index = tab;
})
end
local dependenceMeta
dependenceMeta = {
__index = function(self,k)
-- Allow methods, because OOP
local method = dependenceMeta[k]
if method then return method end
local variable = self.vars[k]
if not variable then
error("Variable "..k.." not found",2)
end return self.vals[k]
end;
__newindex = function(self,k,v)
local variable = self.vars[k]
if not variable then
error("Use :Register() to add stuff",2)
elseif type(v) == "table" then
self.vals[k] = createSubTable(self,k,v)
return updateStuff(self,k,{})
end self.vals[k] = v updateStuff(self,k,{})
end
}
function dependenceMeta:Register(var,value)
local varobject = {func=value,name=var}
self.vars[var] = varobject
table.insert(self.levars,varobject)
if type(value) == "function" then
varobject.needed = getNeededVars(self,value)
self.vals[var] = value(self)
elseif type(value) == "table" then
self.vals[var] = createSubTable(self,var,value)
elseif value then
self.vals[var] = value
end
end
function dependenceMeta:RegisterAll(tab)
for k,v in pairs(tab) do
self:Register(k,v)
end
end
local function DependenceTable()
return setmetatable({
levars = {};
vars = {};
vals = {};
},dependenceMeta)
end
local test = DependenceTable()
test:Register("border",{
x=20; y=50;
height=200;
width=100;
})
test:Register("body",function(self)
return {x=self.border.x+1,y=self.border.y+1,
height=self.border.height-2,
width=self.border.width-2}
end)
test:Register("font",function(self)
local size = (self.body.height+2)-(math.floor((self.body.height+2)/4)+1);
return { size = size; -- Since we use it in the table constructor...
height = size-4; --love.graphics.setNewFont( self.font.size ):getHeight();
-- I don't run this on love, so can't use the above line. Should work though.
}
end)
test:Register("padding",function(self)
local height = math.floor(self.border.height*(2/29))
return { height = height; width = height*3 } -- again dependency
end)
test:Register("text",{input=""}) -- Need this initially to keep input
test:Register("text",function(self)
return { input = self.text.input;
centerHeight = math.ceil(self.body.y+((self.body.height-self.font.height)/2));
left = self.body.x+self.padding.width+self.padding.height;
}
end)
test:Register("backspace",{key = false, rate = 3, time = 0, pausetime = 20, pause = true})
-- Again, didn't use gui.id() on the line below because my lack of LÖVE
test:Register("config",{active=true,devmode=false,debug=false,id=123,type='textbox'})
print("border.x=20, test.text.left="..test.text.left)
test.border = {x=30; y=50; height=200; width=100;}
print("border.x=30, test.text.left="..test.text.left)
test.border.x = 40
print("border.x=40, test.text.left="..test.text.left)
It's a lot of code, but I liked writing it. It gives this nice output:
border.x=20, test.text.left=73
border.x=30, test.text.left=83
border.x=40, test.text.left=93
All properties only get recalculated when one of its dependencies is edited. I made it also work with subtables, which was a bit tricky, but at the end actually seems quite easy. You can edit (for example) the body field by setting it to a completely new table or by setting a field in the already existing table, as seen in the last few lines of the code snippet. When you assign it to a new table, it'll set a metatable on it. You can't use pairs (& co) neither, unless you use 5.2 and can use __pairs.
It might solve your problem. If not, I had fun writing it, so at least it'll always be something positive that I wrote this. (And you have to admit, that's some beautiful code. Well, the way it works, not the actual formatting)
Note: If you're gonna use it, uncomment the love.graphics and gui.id part, as I don't have LÖVE and I obviously had to test the code.
Here's a quick "summary" of my thing's API, as it might be confusing in the beginning:
local hmm = DependenceTable() -- Create a new one
print(hmm.field) -- Would error, "field" doesn't exist yet
-- Sets the property 'idk' to 123.
-- Everything except functions and tables are "primitive".
-- They're like constants, they never change unless you do it.
hmm:Register("idk",123)
-- If you want to actually set a regular table/function, you
-- can register a random value, then do hmm.idk = func/table
-- (the "constructor registering" only happens during :Register())
-- Sets the field to a constructor, which first gets validated.
-- During registering, the constructor is already called once.
-- Afterwards, it'll get called when it has to update.
-- (Whenever 'idk' changes, since 'field' depends on 'idk' here)
hmm:Register("field",function(self) return self.idk+1 end)
-- This errors because 'nonexistant' isn't reigstered yet
hmm:Register("error",function(self) return self.nonexistant end)
-- Basicly calls hmm:Register() twice with key/value as parameters
hmm:RegisterAll{
lower = function(self) return self.field - 5 end;
higher = function(self) return self.field + 5 end;
}
-- This sets the property 'idk' to 5.
-- Since 'field' depends on this property, it'll also update.
-- Since 'lower' and 'higher' depend on 'field', they too.
-- (It happens in order, so there should be no conflicts)
hmm.idk = 5
-- This prints 6 since 'idk' is 5 and 'field' is idk+1
print(hmm.field)
You could use setfenv (if Lua 5.1) to remove the need of 'self.FIELD'. With some environment magic you can have the constructor for 'field' (as an example) just be function() return idk+1 end.
You could make use of metatables, more specific, the __newindex field:
(Well, need to combine it with the __index field, but eh)
function new(x, y, width, height )
local object = {
font = {}, padding = {}, text = {input=''}, -- tables themself are static
-- also I assume text.input will change and has to stay the way it is
}
-- more static data here (yes yes, I know. The code is a bit ugly, but if it works fine...)
object.config = { active = true, devmode = false, debug = false, id = gui.id(), type = 'textbox' }
object.backspace = {key = false, rate = 3, time = 0, pausetime = 20, pause = true}
object.border = { x = x, y = y, width = width, height = height }
-- stuff that has to be calculated from the above variables goes below
local border = object.border
local function calculate()
--border
--body
object.body = { x = border.x+1, y = border.y+1, width = border.width-2, height = border.height-2 }
--font
object.font.size = height-(math.floor(height/4)+1)
object.font.height = love.graphics.setNewFont( object.font.size ):getHeight()
--padding
object.padding.height = math.floor(object.border.height*(2/29))
object.padding.width = object.padding.height*3
--text
object.text.centerHeight = math.ceil(object.body.y+((object.body.height-object.font.height)/2))
object.text.left = object.body.x+object.padding.width+object.padding.height
--backspacing
--config
end
calculate()
local proxy = setmetatable({},{
__index = object; -- proxy.abc returns object.abc (to get width, use proxy.border.width)
__newindex = function(s,k,v)
-- fires whenever 'proxy[k] = v' is done
-- I assume you'll only change x/y/width/height, as other properties are dynamic
-- Doing 'proxy.x = 123' is the same as 'object.border.x = 123' + recalculating
object.border[k] = v -- Actually apply the change
calculate() -- Recalculate the other properties that depends on the above
end;
})
gui.add(object)
return object.config.id
end
You can run code like proxy.x = 12 to edit the X property. All values will be recalculated. It's not the best, but your code a tiny bit annoying to improve. (But hey, if it works fine for you, it's good)
Note: You can only set x, y, width and height. You can get all properties the old way though, e.g. proxy.padding.width (Mind that proxy.x doesn't work. Use proxy.border.x)
It's rather embarrassing to ask this question, since it seems to be so trivial, yet I can't find a solution that works.
I have the following function:
def planner(departure_id, arrival_id):
departure = Stop.get(Stop.id == departure_id)
arrival = Stop.get(Stop.id == arrival_id)
buses = Bus.select().join(RideStopRelationship).join(Stop).where(Stop.id == departure)
for bus in buses:
print bus.line
for stop in bus.stops:
print stop.time, stop.stop.name
based on the following models:
class Stop(BaseModel):
name = CharField()
#lat = FloatField()
#lng = FloatField()
class Bus(BaseModel):
line = IntegerField()
number = IntegerField()
direction = IntegerField()
class RideStopRelationship(BaseModel):
bus = ForeignKeyField(Bus, related_name = "stops")
stop = ForeignKeyField(Stop, related_name = "buses")
time = TimeField()
The crucial line is Bus.select().join(RideStopRelationship).join(Stop).where(Stop.id == departure). I'm trying to get all buses that will stop at both departure and arrival. However, the above query returns all buses that stop at departure. How would I get buses that stop at both 'departure' and 'arrival'?
If I'm making this too complicated (either my models being too complicated, or my query), feel free to correct me.
EDIT:
There's one way that does work:
buses_departure = Bus.select().join(RideStopRelationship).join(Stop).where(Stop.id == departure)
buses_arrival = Bus.select().join(RideStopRelationship).join(Stop).where(Stop.id == arrival)
buses = Bus.select().where(Bus.id << buses_departure & Bus.id << buses_arrival)
but it's rather long for what should be a simply query...
You might try something like this:
departure = Stop.get(...)
arrival = Stop.get(...)
query = (Bus
.select(Bus)
.join(RideStopRelationship)
.where(RideStopRelationship.stop << [departure, arrival])
.group_by(Bus)
.having(fn.Count(Bus.id) == 2))
Unrelated, but one thing to note is that due to the way python evaluates operators, you need to put parentheses around your in queries:
buses = Bus.select().where(
(Bus.id << buses_departure) &
*Bus.id << buses_arrival))
I have a problem with a function in matlab. This specific function is for filtering light signals. As you can see below I added the coding I’ve used in the function and in the while loop itself. The code is written for a NXT Lego robot.
Is there any tip how to get the count variable ( i = i + 1 ) to work in the function, so we can plot Light(i)? Because we’re getting a bunch of error messages when we try different codes to make it work.
function [light] = filter_func( i)
lightI(i) = GetLight(SENSOR_3);
if i==1
light(i)=lightI(i)
elseif i==2
light(i) = 0.55*lightI(i) + 0.45*lightI(i-1)
else
light(i) = 0.4*lightI(i) + 0.3*lightI(i-1) + 0.3*lightI(i-2);
end
end
i=1
while true
lightI(i) = GetLight(SENSOR_3); % Get’s a lightvalue between 0 and 1024.
if i>2
light =filter_func(i)
light=round(light);
else
light(i) = GetLight(SENSOR_3);;
end
i=1+i
plot(light(end-90:end), 'r-');
title('Lightvalue')
axis([0 100 0 1023]) ;
end
You probably mainly get errors because you are not allowed to mix script and functions like this in MATLAB (like you are in Python).
Your filter function is only used when i>2 so why are you doing the first 2 tests? It seems like you want lightI as a global variable, but that is not what you have done. The lightI inside the function is not the same as the one in the while loop.
Since your while loop runs forever, maybe you don't need to worry about updating the plot the first two times. In that case you can do this:
filter = [0.4 0.3 0.3]';
latest_filtered_light = nan(90,1);
lightI = [];
p = plot(latest_filtered_light, 'r-');
title('Lightvalue')
axis([0 100 0 1023]) ;
while True
lightI(end+1,1) = rand*1024; % Get’s a lightvalue between 0 and 1024.
if i>=3
new_val = lightI(end-2:end,1)'*filter;
latest_filtered_light = [latest_filtered_light(2:end);...
new_val];
set(p, 'ydata', latest_filtered_light)
drawnow
end
end
I think it is an important point to not call plot every time - at least if you are the least concerned about performance.
SELECT ica.CORP_ID, ica.CORP_IDB, ica.ITEM_ID, ica.ITEM_IDB,
ica.EXP_ACCT_NO, ica.SUB_ACCT_NO, ica.PAT_CHRG_NO, ica.PAT_CHRG_PRICE,
ica.TAX_JUR_ID, ica.TAX_JUR_IDB, ITEM_PROFILE.COMDTY_NAME
FROM ITEM_CORP_ACCT ica
,ITEM_PROFILE
WHERE (ica.CORP_ID = 1000)
AND (ica.CORP_IDB = 4051)
AND (ica.ITEM_ID = 1000)
AND (ica.ITEM_IDB = 4051)
AND ica.EXP_ACCT_NO = ITEM_PROFILE.EXP_ACCT_NO
I'm trying basically say since the exp account code is '801500' then the Name should return "Miscellaneous Medic...".
It seems as if what you are showing is not possible. Have you edited the data in the editor??? You are joining using ica.EXP_ACCT_NO = ITEM_PROFILE.EXP_ACCT_NO . Therefore, every entry with EXP_ACCT_NO = 801500, should also have the same COMDTY_NAME.
However, it could be the case that your IDs are not actually numbers and that they are strings with whitespace (801500__ vs 801500 ). But since you are not performing a left-outer join, it would also mean you have an entry in ITEM_PROFILE with the same whitespace.
You also need to properly normalize your table data (unless this is a view) but it still means you have erroneous data.
Try to perform the same query, but using the TRIM function to remove whitespace: https://stackoverflow.com/a/6858168/1688441 .
Example:
SELECT ica.CORP_ID, ica.CORP_IDB, ica.ITEM_ID, ica.ITEM_IDB,
ica.EXP_ACCT_NO, ica.SUB_ACCT_NO, ica.PAT_CHRG_NO, ica.PAT_CHRG_PRICE,
ica.TAX_JUR_ID, ica.TAX_JUR_IDB, ITEM_PROFILE.COMDTY_NAME
FROM ITEM_CORP_ACCT ica
,ITEM_PROFILE
WHERE (ica.CORP_ID = 1000)
AND (ica.CORP_IDB = 4051)
AND (ica.ITEM_ID = 1000)
AND (ica.ITEM_IDB = 4051)
AND trim(ica.EXP_ACCT_NO) = trim(ITEM_PROFILE.EXP_ACCT_NO);