I want to load data from CSV file like this:
var format = new CSVFormat('.', ' ');
IVersatileDataSource source = new CSVDataSource(filename, false, format);
var data = new VersatileMLDataSet(source); ...
Then I have two options:
Use EncogModel
var model = new EncogModel(data);
model.SelectMethod(data, MLMethodFactory.TypeFeedforward); ...
Make own network
var network = new BasicNetwork();
network.AddLayer(new BasicLayer(null, true, 11));
network.AddLayer(new BasicLayer(new ActivationSigmoid(), true, 8));
network.AddLayer(new BasicLayer(new ActivationTANH(), true, 5));
...
IMLDataSet trainingSet = new BasicMLDataSet(input, output);
I don't know how to set number of layers, neurons and activation functions with first option (Encog Model). All I get is some default feedforward network with one hidden layer only.
I don't know how can get easily input and output arrays separately for my own network (second option) from VersatileMLDataSet. I can get whole array (input + output), but there must be a way how to get only input array or output array.
I found answer in documentation (Encog Method & Training Factories, page 75), with EncogModel is possible customize network like this:
var methodFactory = new MLMethodFactory();
var method = methodFactory . Create(
MLMethodFactory .TYPEFEEDFORWARD,
”?:B−>SIGMOID−>4:B−>SIGMOID−>?”,
2,
1);
The above code creates a neural network with two input neurons and one
output neuron. There are four hidden neurons. Bias neurons are placed
on the input and hidden layers. As is typical for neural networks,
there are no bias neurons on the output layer. The sigmoid activation
function is used between both the input and hidden neuron, as well
between the hidden and output layer. You may notice the two question
marks in the neural network architecture string. These will be filled
in by the input and output layer sizes specified in the create method
and are optional. You can hard-code the input and output sizes. In
this case the numbers specified in the create call will be ignored.
Related
I have a FeatureCollection made up of many (100-200) polygons ('ftr_polygons'). I also have an ImageCollection made up of monthly median Landsat8 bands and indices ('byMonth'). I want to ReduceRegions and save a median (or mean) spatial average from each polygon in the FeatureCollection. End goal is to export to csv a timeseries of monthly mean bands/indices within each polygons over multiple years (2013-2019).
With the code below, I am able to do this for ~1 year, but any more than that, and I get an error: 'FeatureCollection (Error) Computation timed out’. Is there a better way to do this?
// define the function that will grab median (or mean) spatial reductions for each polygon, for each month
var extractdata = function(medianImage,ftr_polygons) {
var date_start = ee.Date(medianImage.get('system:time_start')).format("YYYY-MM"); // get date as string to append to each property
// spatial MEDIAN
ftr_polygons = medianImage.reduceRegions({ // create feature collection with new properties, bands for each month, uniquely named
collection: ftr_polygons,
reducer: ee.Reducer.median(),
scale: 30,
tileScale: 1}); // tile scale
var ftr_polygons_propnames = ftr_polygons.first().propertyNames(); // get property names first
var ftr_polygons_newnames = ftr_polygons_propnames.replace('NDVI_median',
ee.String('NDVI_median_').cat(date_start)); //replace property names with band+date
ftr_polygons_newnames = ftr_polygons_newnames.replace('EVI_median',
ee.String('EVI_median_').cat(date_start)); //replace property names with band+date
ftr_polygons_newnames = ftr_polygons_newnames.replace('NIRv_median',
ee.String('NIRv_median_').cat(date_start)) ; //replace property names with band+date
ftr_polygons = ftr_polygons.map(function(f) {return f.select(ftr_polygons_propnames,ftr_polygons_newnames)});
return ftr_polygons;
};
// apply the function over ImageCollection byMonth, beginning with feature collection ftr_polygons
var ftr_polygons = ee.FeatureCollection(byMonth.iterate(extractdata,ftr_polygons));
// remove geometry on each feature before printing or exporting
var myproperties=function(feature){
feature=ee.Feature(feature).setGeometry(null);
return feature;
};
var ftr_polygon_export = ftr_polygon.map(myproperties)
print(ftr_polygon_export.limit(1), 'For export w monthly properties');
Maybe this answer: https://stackoverflow.com/a/48412324/12393507 alludes to a better way:
The same approach can be used with reduceRegions() as well, mapping over images and then over regions. However, you will have to map over the resulting features to set dates.
I would appreciate more info on this approach.
Thanks.
For computationally intensive operations that will run for a long time you should always export your results instead of visualizing/printing them.
For more info read through this section of the debugging page in the Earth Engine manual.
I use torch-caffe-binding to load caffe model to torch, and the example shows that
require 'caffe'
net = caffe.Net('deploy.prototxt', 'bvlc_alexnet.caffemodel', 'test')
input = torch.FloatTensor(10,3,227,227)
output = net:forward(input)
gradOutput = torch.FloatTensor(10,1000,1,1)
gradInput = net:backward(input, gradOutput)
but when i use backward to train the net,no matter how many times to train it,the output doesn't change when input same data.so how to train the caffe model,thanks in advance.
add:
i'm sorry that i didn't say clearly.my code like this:
caffenet = caffe.Net('test.prototxt','test2.caffemodel','train')
caffe_output = caffenet:forward(input)
local predParts=netPred:forward(caffe_output)
netPred:backward(caffe_output, gradPred)
caffenet:backward(input,g)
netPred is a torch net,andcaffenet is a net that remove the last loss layer.but i set the second last layer loss_weight=1and console shows all layers like thisip1 needs backward computation.,but i check the output never changed when i use backward.and even i add the loss layer it also didn't work.
I would like to add to my plot a text or a legend box with comments.
At the moment my legend is plot at northeastoutside and i would like to add the new legend (or textbox) to the position southeastoutside.
Thanks!
Lacking more information about your case:
To the best of my knowledge one axes object can only have a single legend object. You can create a second legend with a second axes object. Each legend will only list data elements associated with each axes. Adapted from Matlab Newsgroup thread
a = [1:0.01:2*pi]; %create sample data
b = sin(a);
linehandle1 = line(a,b); %creates a line plot with handle object
axeshandle1 = gca; % gets the handle for the axes object just created
legendhandle1 = legend('y = sin(x)', 'location', 'northeastoutside'); %makes first legend
axeshandle2 = axes('Position',get(axeshandle1,'Position'),'xlim',get(axeshandle1,'xlim'),'ylim',get(axeshandle1,'ylim'),'Visible','off','Color','none'); %makes invisible axes with same position and scaling
linehandle2 = line(pi/2,1,'Color','r','Marker','o','Parent',axeshandle2); %puts data set on 2nd axes
linehandle3 = line(pi,0,'Color','b','Marker','x','Parent',axeshandle2);
legend_handle2 = legend('peak','zero','location','southeastoutside'); %creates legend to go with 2nd axes
If you just want text in that 2nd box, not necessarily legend info or data labels, you can play around with annotation as described above. This has the advantage of being simpler to call, but maybe harder to get the exact position/result you want. There are a large number of property options that can be adjusted to get the desired appearance. A few are shown in the example. It may be there are easier ways to set the size/position based on the legendhandle.
a = [1:0.01:2*pi]; %create sample data
b = sin(a);
plot(a,b);
legendhandle = legend('y = sin(x)','location','northeastoutside');
annotation('textbox',[0.875 0.1 0.1 0.1],'string','my text','edgecolor','k','linewidth',1,'fitboxtotext','off');
I'm trying to do some re-factoring on my charts to make them re-usable using this as a guide: http://bost.ocks.org/mike/chart/
I'm having problems drawing the lines in my multi-line graph though - specifically passing the data to the x and y values. If I hard code the element names it works, but if I try to use the xValue and yValue objects this does not work. I'm assuming that this is because I'm trying to call a function within the parameter of an other object, but I'm not sure how to get around this. In the exmaple Mike uses d[0] and d[1], but this won't work with JSON data (or I'm not sure how to make it work).
I've posted this JSFiddle so you can see the code. The problem lines are 125 to 131 which in turn is being called from line 165.
var main_line = d3.svg.line()
.interpolate("cardinal")
// Hard coding the elements works
//.x(function(d) { return main_x(d.date); })
//.y(function(d) { return main_y(d.buildFixTime); });
// Passing xValue and yValue does not work
.x(function(d) { return main_x(xValue); })
.y(function(d) { return main_y(yValue); });
http://jsfiddle.net/goodspeedj/fDyLY/
Thank you in advance.
You need to redefine your accessor method within .x() and .y(). The accessor method defines the way that a datum is pulled out of the data that is bound to the selection that you call the line generator on.
Suppose you have a relatively flat data structure such as the following.
data = [{x : 1, y : 2}, {x:1, y:3}, {x:4, y:5}];
You then bind the data to a selection with the following statement
d3.select("body").datum(data).append("path").attr("d",lineGenerator);
Quite a bit is going on underneath this statement. I'll give you a bit more of a walkthrough after showing you a commonly used example.
The important aspect to understand is that similarly to other calls in d3 such as
var exampleRectangles = d3.select("body")
.data(data).enter()
.append("rect")
.attr("width",2)
.attr("height", 3)
.attr("x",function(datum){return datum.x}) // pay attention to this line
.attr("y",0);
d3 is implicitly iterating over each element in your data. For each datum in your data array, in this case there is a total of three datum, you are going to add a rectangle to the dom.
In the line that I tell you to pay attention to you notice that you're defining an anonymous (unnamed) function. What is that datum parameter coming from? It's implicitly being passed to your anonymous function.
So each rectangle has it's own corresponding datum {x : 1, y : 2}, {x:1, y:3}, {x:4, y:5} respectively. Each rectangle's x coordinate is defined by the respective datum.x attribute. Under the sheets, d3 is implicitly looping over the data array that you've defined. A similar approach to the example d3 code could be written as above.
for (var i = 0; i < data.length; i++)
{
d3.select("body").append("rect")
.attr("width",2)
.attr("height", 3)
.attr("x",data[i].x)
.attr("y",0);
}
This follows from the notion of data driven documents (d3). For each item added (a rectangle in the above example a piece of data is tied to it. In the above example you see that there is something kind of similar to your .x() and .y() accessor functions :
.attr("x",function(datum){return datum.x})
This function is telling d3 how to filter over the total datum that's being passed to the .attr() accessor method.
So, you need to determine which data you need to get a hold of to make your .attr("d", lineGenerator)call make sense. The difference between your.datum(data)call and the typical.data(data)call is that instead of parceling the data that's being passed to.data(data)`, the whole array is given as a single piece of data to the line generator function (similar to main_line(data), wherein it will again implicitly loop over the points to construct your path.
So, what you need to do is determine what a single datum will be defined as for your function to operate on.
I'm not going to define that as I don't seem to know quite which information you are operating on, but I would hazard a guess at something like.
.x(xAccessor)
.y(yAccessor)
function xAccessor(datum)
{
return xScale(datum._id.month);
}
function yAccessor(datum)
{
return yScale(datum.buildFixTime);
}
The way you have it set up, xValue and yValue are functions; you have to actually execute them on something to get a value back.
.x(function(d) { return main_x( xValue(d) ); })
.y(function(d) { return main_y( yValue(d) ); });
If you weren't using a scale, you could use
.x(xValue)
.y(yValue);
but only because if you pass in a function d3 executes it for you with the data as a parameter. And that only works for d3 methods that expect functions as possible input -- the scale functions expect data values as input.
I wrote a long piece work for another user last week that you may find useful, explaining methods that accept functions as parameters.
I'm writing a trivia game app in C# that writes data to a binary file, then reads the data from the file into six labels. The six labels are as follows:
lblQuestion // This is where the question text goes.
lblPoints // This is where the question points goes.
lblAnswerA // This is where multiple choice answer A goes.
lblAnswerB // This is where multiple choice answer B goes.
lblAnswerC // This is where multiple choice answer C goes.
lblAnswerD // This is where multiple choice answer D goes.
Here is the code for writing to the binary file:
{
bw.Write(Question);
bw.Write(Points);
bw.Write(AnswerA);
bw.Write(AnswerB);
bw.Write(AnswerC);
bw.Write(AnswerD);
}
Now for the code to read data from the file into the corresponding labels:
{
FileStream fs = File.OpenRead(ofd.FileName);
BinaryReader br = new BinaryReader(fs);
lblQuestion.Text = br.ReadString();
lblPoints.Text = br.ReadInt32() + " points";
lblAnswerA.Text = br.ReadString();
lblAnswerB.Text = br.ReadString();
lblAnswerC.Text = br.ReadString();
lblAnswerD.Text = br.ReadString();
}
The Question string reads correctly into lblQuestion.
The Points value reads correctly into lblPoints.
AnswerA, AnswerB, and AnswerC DO NOT read into lblAnswerA, lblAnswerB and lblAnswerC respectively.
lblAnswerD, however, gets the string meant for lblAnswerA.
Looking at the code for reading data into the labels, is there something missing, some sort of incremental value that needs to be inserted into the code in order to get the strings to the correct labels?