How to calculate SAVI with MODIS in Google Earth Engine (Getting Error: Image.select: Pattern 'B2' did not match any bands.) - gis

I am trying to calculate SAVI vegetation index using MODIS data. But I am getting an error showing:
Image.select: Pattern 'B2' did not match any bands.
Code:
countries = ee.FeatureCollection("USDOS/LSIB_SIMPLE/2017")
canada = countries.filter(ee.Filter.eq("country_na", "Canada"))
image = ee.ImageCollection("MODIS/061/MOD09A1")\
.filterDate('2017-01-01','2017-12-31')\
.filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE',10))\
.filterBounds(canada)\
.median()\
.clip(canada)
savi = image.expression(
'1.5*((NIR-RED)/(NIR+RED+0.5))',{
'NIR':image.select('B2'),
'RED':image.select('B1')
}).rename('savi')
saviVis = {'min':0.0, 'max':1, 'palette':['yellow', 'green']}
Map = geemap.Map()
Map.addLayer(savi, saviVis, 'SAVI')
Map
Why am I getting this error? Isn't B1 designated to Red and B2 to NIR?

The general thing to do when you hit this type of problem is to start examining the dataset for what is actually there — how many images are you matching, what properties and bands those images have, etc. I found two problems:
Your filter criteria matched zero images. Therefore the collection is empty, and therefore the median() image from that collection has no bands at all. (You can check this by putting the collection in a variable and printing the size() of it.) You will need to adjust the criteria.
It seems that the main reason they didn't match is that the images in MODIS/061/MOD09A1 do not have a CLOUDY_PIXEL_PERCENTAGE property.
The band names for MODIS/061/MOD09A1 are not B1, B2, ... but sur_refl_b01, sur_refl_b02 and so on. You can see this with the Inspector in the Earth Engine Code Editor, or on the dataset description page.
Perhaps you were working from information about a different dataset?
With the two problems above fixed, your code produces some results. This is the (JS) version I produced while testing (Code Editor link):
var countries = ee.FeatureCollection("USDOS/LSIB_SIMPLE/2017");
var canada = countries.filter(ee.Filter.eq("country_na", "Canada"));
var images = ee.ImageCollection("MODIS/061/MOD09A1")
.filterDate('2017-01-01','2017-12-31')
// .filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE',10))
.filterBounds(canada);
// print(images);
var image = images.median().clip(canada);
Map.addLayer(canada);
Map.addLayer(image);
var savi = image.expression(
'1.5*((NIR-RED)/(NIR+RED+0.5))',{
'NIR':image.select('sur_refl_b02'),
'RED':image.select('sur_refl_b01')
}).rename('savi');
var saviVis = {'min':0.0, 'max':1, 'palette':['yellow', 'green']};
Map.addLayer(savi, saviVis, 'SAVI')

Related

append dataframe in specific cell

I am trying to grab data from a mysql database and put it in an excel template ( with macro's).
The template has mutiple sheets.I want to put the data in a specific sheet and specific cell ( B2 ) since the sheet already contains data.
The code i am using is:
wb= openpyxl.load_workbook('C:/Users/Olav/Desktop/Xenos/Nieuw.xlsx')
ws = wb['Dump Pickloc - del. web']
picklocaties = "SELECT Artikelnummer, Locatie,PICKZONE FROM picklocaties WHERE PICKZONE in ('BASIS','HL')"
df = pd.read_sql(sql=picklocaties, con=mydb)
rows = dataframe_to_rows(df)
for r in dataframe_to_rows(df, index=False, header=False):
ws.append(r)
I tryed using to_excel but that just deletes everything.
The template in which i am putting the data looks like This.
It would be great if this code would work but it does not have that option:
for r in dataframe_to_rows(df, index=False, header=False, startrow=1, startcol=1):
ws.append(r) \
Woah i'm half way there, woah living on prayer.
This codes gets me halfway. I get the columns now where i want without messing up the rest. But for some reason the rest of the data is not shown.
for col, text in enumerate(df, start=2):
ws.cell(column=col,row=2, value=text)

anova_test not returning Mauchly's for three way within subject ANOVA

I am using a data set called sleep (found here: https://drive.google.com/file/d/15ZnsWtzbPpUBQN9qr-KZCnyX-0CYJHL5/view) to run a three way within subject ANOVA comparing Performance based on Stimulation, Deprivation, and Time. I have successfully done this before using anova_test from rstatix. I want to look at the sphericity output but it doesn't appear in the output. I have got it to come up with other three way within subject datasets, so I'm not sure why this is happening. Here is my code:
anova_test(data = sleep, dv = Performance, wid = Subject, within = c(Stimulation, Deprivation, Time))
I also tried to save it to an object and use get_anova_table, but that didn't look any different.
sleep_aov <- anova_test(data = sleep, dv = Performance, wid = Subject, within = c(Stimulation, Deprivation, Time))
get_anova_table(sleep_aov, correction = "GG")
This is an ideal dataset I pulled from the internet, so I'm starting to think the data had a W of 1 (perfect sphericity) and so rstatix is skipping this output. Is this something anova_test does?
Here also is my code using a dataset that does return Mauchly's:
weight_loss_long <- pivot_longer(data = weightloss, cols = c(t1, t2, t3), names_to = "time", values_to = "loss")
weight_loss_long$time <- factor(weight_loss_long$time)
anova_test(data = weight_loss_long, dv = loss, wid = id, within = c(diet, exercises, time))
Not an expert at all, but it might be because your factors have only two levels.
From anova_summary() help:
"Value
return an object of class anova_test a data frame containing the ANOVA table for independent measures ANOVA. However, for repeated/mixed measures ANOVA, it is a list containing the following components are returned:
ANOVA: a data frame containing ANOVA results
Mauchly's Test for Sphericity: If any within-Ss variables with more than 2 levels are present, a data frame containing the results of Mauchly's test for Sphericity. Only reported for effects that have more than 2 levels because sphericity necessarily holds for effects with only 2 levels.
Sphericity Corrections: If any within-Ss variables are present, a data frame containing the Greenhouse-Geisser and Huynh-Feldt epsilon values, and corresponding corrected p-values. "

In Google Earth Engine: Most efficiently reduceRegions over each image in ImageCollection, saving mean as a Feature property?

I have a FeatureCollection made up of many (100-200) polygons ('ftr_polygons'). I also have an ImageCollection made up of monthly median Landsat8 bands and indices ('byMonth'). I want to ReduceRegions and save a median (or mean) spatial average from each polygon in the FeatureCollection. End goal is to export to csv a timeseries of monthly mean bands/indices within each polygons over multiple years (2013-2019).
With the code below, I am able to do this for ~1 year, but any more than that, and I get an error: 'FeatureCollection (Error) Computation timed out’. Is there a better way to do this?
// define the function that will grab median (or mean) spatial reductions for each polygon, for each month
var extractdata = function(medianImage,ftr_polygons) {
var date_start = ee.Date(medianImage.get('system:time_start')).format("YYYY-MM"); // get date as string to append to each property
// spatial MEDIAN
ftr_polygons = medianImage.reduceRegions({ // create feature collection with new properties, bands for each month, uniquely named
collection: ftr_polygons,
reducer: ee.Reducer.median(),
scale: 30,
tileScale: 1}); // tile scale
var ftr_polygons_propnames = ftr_polygons.first().propertyNames(); // get property names first
var ftr_polygons_newnames = ftr_polygons_propnames.replace('NDVI_median',
ee.String('NDVI_median_').cat(date_start)); //replace property names with band+date
ftr_polygons_newnames = ftr_polygons_newnames.replace('EVI_median',
ee.String('EVI_median_').cat(date_start)); //replace property names with band+date
ftr_polygons_newnames = ftr_polygons_newnames.replace('NIRv_median',
ee.String('NIRv_median_').cat(date_start)) ; //replace property names with band+date
ftr_polygons = ftr_polygons.map(function(f) {return f.select(ftr_polygons_propnames,ftr_polygons_newnames)});
return ftr_polygons;
};
// apply the function over ImageCollection byMonth, beginning with feature collection ftr_polygons
var ftr_polygons = ee.FeatureCollection(byMonth.iterate(extractdata,ftr_polygons));
// remove geometry on each feature before printing or exporting
var myproperties=function(feature){
feature=ee.Feature(feature).setGeometry(null);
return feature;
};
var ftr_polygon_export = ftr_polygon.map(myproperties)
print(ftr_polygon_export.limit(1), 'For export w monthly properties');
Maybe this answer: https://stackoverflow.com/a/48412324/12393507 alludes to a better way:
The same approach can be used with reduceRegions() as well, mapping over images and then over regions. However, you will have to map over the resulting features to set dates.
I would appreciate more info on this approach.
Thanks.
For computationally intensive operations that will run for a long time you should always export your results instead of visualizing/printing them.
For more info read through this section of the debugging page in the Earth Engine manual.

How can I search pipeline with another pipeline value on google cloud dataflow

I would like to search text which includes specified word from stream data with google cloud dataflow.
In detail, I will deal with following two stream.
stream A: element of stream is "word"
stream B: element of stream is "text". and each text consists of "word". This text may have "word" on stream A
Many "text" flow into stream B frequently. On the other hand, "word" flow into stream A occasionally.
When "word" flow into stream A, I would like to search "text" which has "word" and flow into stream B after 5 minutes ago.
Example
time stream A : stream B
00:01 - this is an apple
00:02 - this is an orange
00:03 - I have an apple
00:04 apple <= "this is an apple" and "I have an apple" are found
00:05 this <= "this is an apple" and "this is an orange" are found
Can I search text with google cloud dataflow?
If I understand your question correctly, there are multiple ways to achieve something like what you want. I will describe two variations.
The basic idea in my example code is to use an inner join and SlidingWindows of five minutes. You can implement the join using ParDo side inputs or CoGroupByKey, depending on your data sizes.
Here is how you set up your inputs and windowing:
PCollection<String> streamA = ...;
PCollection<String> streamB = ...;
PCollection<String> windowedStreamA = streamA.apply(
Window.into(
SlidingWindows.of(Duration.standardMinutes(5)).every(...)));
PCollection<String> windowedStreamB = streamB.apply(
Window.into(
SlidingWindows.of(Duration.standardMinutes(5)).every(...)));
You may want to adjust the size of windows or period to meet your specification & performance needs.
Here is a sketch of how to do the join with side inputs. This will iterate over the entire five minute window of streamB for each element of streamA, so performance will suffer if windows get large.
PCollectionView<Iterable<String>> streamBview = streamB.apply(View.asIterable());
PCollection<String> matches = windowedStreamA.apply(
ParDo.of(new DoFn<String, String>() {
#Override void processElement(ProcessContext context) {
for (String text : context.sideInput()) {
if (split(text).contains(context.element())) {
context.output(text);
}
}
}
});
Here is a sketch of how to do this with CoGroupByKey by pre-splitting the text and joining each keyword with the lines that contain that keyword. There is similar logic in the TfIdf example included with the SDK.
PCollection<KV<String, Void>> keyedStreamA = windowedStreamA.apply(
MapElements
.via(word -> KV.of(word, null))
.withOutputType(new TypeDescriptor<KV<String, Void>>() {}));
PCollection<KV<String, String>> keyedStreamB = windowedStreamB.apply(
FlatMapElements
.via(text -> split(text).forEach(word --> KV.of(word, text))
.withOutputType(new TypeDescriptor<KV<String, String>>() {}));
TupleTag<Void> tagA = new TupleTag<Void>() {};
TupleTag<String> tagB = new TupleTag<String>() {};
KeyedPCollectionTuple coGbkInput = KeyedPCollectionTuple
.of(tagA, keyedStreamA)
.and(tagB, keyedStreamB);
PCollection<String> matches = coGbkInput
.apply(CoGroupByKey.create())
.apply(FlatMapElements
.via(result -> result.getAll(tagB))
.withOutputType(new TypeDescriptor<String>()));
The best approach will depend on your data. If you are OK with getting more matches than just the last five minutes you can tune the amount of data duplication in windows by enlarging your sliding windows and having a larger period. You can also use triggers to tune when output is produced.

Adobe After Effects: Keep "Expression-Relations" when duplicating multiple layers

just wanted to ask, whether there is a way to keep the relations of expressions going when duplicating layers.
E.g. I have two layers, "LayerA" and "LayerB". Now I have an expression going on in "LayerB" saying, that its position always equals the position of "LayerA".
Now when I duplicate those two and get "LayerA 2" and "LayerB 2" I want the expression in "LayerB 2" to reference to "LayerA 2"'s position rather than "LayerA"'s position!
While it is no problem to simply change the expression when there is only one of them, it gets quite hard when you have multiple expressions going on ...
You might end up wanting to organize your comp differently, but, given your example (and exactly those name lengths), this position expression will work to find the appropriate 'target layer':
//base name to work from:
baseName = "Layer";
//length of that:
nameLen = baseName.length;
//this layer's name:
myName = thisLayer.name;
if (myName.length == nameLen) {
//if they are the same, then it is the original
// (non-duplicated) version
thisComp.layer("LayerA").transform.position;
} else {
//get tail string, the space and number:
tailStr = myName.substring(nameLen+1, myName.length);
//build new target layer name with "A":
targetName = myName.substring(0, (nameLen)) + "A" + tailStr
//new line pointing to target layer:
thisComp.layer(targetName).transform.position;
}