I have the following flux query that aggregates on a monthly basis
from(bucket: "some-bucket")
|> range(start: v.timeRangeStart, stop: v.timeRangeStop)
|> filter(fn: (r) => r._measurement == "some-value" and r._field == "some-other-value")
|> aggregateWindow(every: 1mo, fn: sum)
that gives the following bar chart
As you can see the bars are very thin. I would like them to be one month wide. Is there a way to manually set this or a different way to construct the query to get Grafana to detect this?
Thanks in advance for any help.
It seems that this has to do with the time spacing of the very last record. A workaround for me was to use the _start column of groups (instead of the default _stop) to restore the _time column.
Try this:
from(bucket: "some-bucket")
|> range(start: v.timeRangeStart, stop: v.timeRangeStop)
|> filter(fn: (r) => r._measurement == "some-value" and r._field == "some-other-value")
|> aggregateWindow(every: 1mo, fn: sum, timeSrc: "_start")
See https://stackoverflow.com/a/66107599/1616948 for details.
Related
I have a dataframe "d1" from a big MySQL table. I need find there an unused columns (which contains only NA or empty strings).
(see question Find columns with all missing values ).
This seems to work fine:
allmisscols <- apply(d1,2, function(x)all(is.na(x)));
colswithallmiss <-names(allmisscols[allmisscols>0]);
cat( colswithallmiss,sep="\n");
...
allmisscols <- apply(d1,2, function(x)all(x==''));
colswithallmiss <-names(allmisscols[allmisscols>0]);
cat( colswithallmiss,sep="\n");
...
although the second one gives also "NA" among the column names; i don't understand why.
But when I try to combine them:
allmisscols <- apply(d1,2, function(x)all(is.na(x)||x=='') );
colswithallmiss <-names(allmisscols[allmisscols>0]);
print("the columns with all values missing");
print(colswithallmiss);
I see a column in result that actually contain a value in my table!
The same gives following:
library(stringr);
sapply(d1, function(x)all(any(is.na(x)||(str_trim(x)==""))))
So my questions are:
Why I've got such unexpected results?
How can I get the list of column which contains only empty OR N/A values?
Try this:
allmisscols <- sapply(dt, function(x) all(is.na(x) | x == '' ))
Note: You've used OR as double '||' trying making it a single one. Read this SO post: Boolean operators && and ||
In F#, is it possible for a function to take one required parameter and one or more optional parametes depending on context? In the following toy example, whisk initially takes eggYolks as its only parameter but, in the very next step, it takes the output of the initial step plus granulatedSugar and marsalaWine. Is this possible and how do I feed the additional ingredients to tiramisu and print out both steps to the console?
module Tiramisu =
// http://www.lihaoyi.com/post/WhatsFunctionalProgrammingAllAbout.html
open System
// Ingredients.
let eggYolks = "70<g> of egg yolks."
let granulatedSugar = "100<g> of granulated sugar."
let marsalaWine = "120<ml> of sweet marsala wine."
let whisk ingredient = printf "Whisk %s\t" ingredient
let tiramisu ingredients =
ingredients
|> whisk // eggYolks only.
// |> whisk // plus granulatedSugar and marsalaWine.
[<EntryPoint>]
tiramisu eggYolks
// tiramisu (eggYolks granulatedSugar marsalaWine)
Summary: You should write whisk to take a list. See below for the full explanation, which starts with the wrong approach, explains why it's the wrong approach, and then moves to the right approach.
Long explanation:
The question you're asking is whether you could write the function whisk to take multiple things to be whisked, e.g. you're asking whether a whisk function could look like:
let whisk item1 maybeItem2 maybeItem3 =
printfn "Whisking %A" item1
match maybeItem2 with
| None -> ()
| Some item -> printfn "Also whisking %A" item
match maybeItem3 with
| None -> ()
| Some item -> printfn "Also whisking %A" item
But this design has some problems. For one thing, this function's type signature is inconvenient: the first parameter is an ingredient, but the second and third parameters might be ingredients (they're actually Options). In other words, if you had specified the types of the parameters in your function, they would have looked like:
type Ingredient = string // For this example
let whisk (item1 : Ingredient) (maybeItem2 : Ingredient option) (maybeItem3 : Ingredient option) =
// ... function body goes here ...
Why is this inconvenient? Well, if you only wanted to whisk a single thing, you'd have to call this function as whisk eggYolks None None. (Calling it without the two None parameters would get you a partially-applied function, which is a different topic). And another inconvenience: this is limited to just three items; if you wanted to whisk four items, you'd have to change the function signature, and then you'd have to change everywhere it was called to pass four parameters by adding an extra None to each call.
Also, this example function doesn't actually return anything, for simplicity. If it did return something, it gets even more complicated. For example, if you're coming from an imperative language like C#, you might try writing this:
type Ingredient = string // For this example
let whisk (item1 : Ingredient) (maybeItem2 : Ingredient option) (maybeItem3 : Ingredient option) =
printfn "Whisking %A" item1
let mutable mixtureSoFar = item1
match maybeItem2 with
| None -> ()
| Some item ->
printfn "Also whisking %A" item
mixtureSoFar <- mixtureSoFar + item
match maybeItem3 with
| None -> ()
| Some item ->
printfn "Also whisking %A" item
mixtureSoFar <- mixtureSoFar + item
mixtureSoFar
But that's ugly. When your F# code starts looking ugly, that's usually a sign that your design is off, somehow. For example, maybe you could let the whisk function take a list of ingredients, instead of trying to pass multiple parameters where some of them might be None. E.g., the whisk function would instead look like:
let whisk (items : Ingredient list) =
// ... function body goes here ...
And then you'd call it like this:
let whiskedEggYolks = whisk [eggYolks]
let mixture = whisk [whiskedEggYolks; granulatedSugar; marsalaWine]
What would that function look like inside? Well, it would probably apply some transformation to each ingredient, then some combining function to combine all those ingredients together into a single result. In F#, "apply some transformation to each item" is called map, and "apply some combining function to combine multiple items into a single one" is either fold or reduce. (I'll explain the difference between fold and reduce below). Here, I think you'd want reduce, since whisking an empty bowl doesn't make sense. So our whisk function becomes:
let whisk (ingredients : Ingredient list) =
ingredients
|> List.map (fun x -> sprintf "%s, whisked" x)
|> List.reduce (fun a b -> sprintf "%s, plus %s" a b)
When you whisk "70<g> of egg yolks", you get "70<g> of egg yolks, whisked". Then when you whisk that together with "100<g> of granulated sugar" and "120<ml> of sweet marsala wine", you get the output:
"70<g> of egg yolks, whisked, plus 100<g> of granulated sugar, whisked, plus 120<ml> of sweet marsala wine, whisked"
And yet your function is beautifully simple (just three lines to handle any number of ingredients!) and you didn't have to write any of the list-handling code, since that was taken care of by the standard F# core library functions List.map and List.reduce. That sort of elegance is what you should be aiming for when you do functional programming.
Fold and reduce
I said I'd explain the difference between fold and reduce. The main difference is whether you expect to be dealing with empty collections sometimes. The reduce function requires that there be at least one item in the collection you're reducing, and doesn't need an initial value since the first item of the collection is taken as the initial value. But because reduce needs the first item of the collection to set its initial value, it will throw an exception if it is passed an empty collection, because there's no way for it to know what value to use. (F# deliberately avoids null, for good reason -- so it's not always possible to determine a good value for an empty collection). Whereas fold requires you to specify an explicit initial value, but it is okay with an empty collection, because if you pass an empty collection then it just returns the default value. E.g.,
let zeroInts = []
let oneInt = [1]
let twoInts = [1; 2]
let add x y = x + y
zeroInts |> List.reduce add // Error
oneInt |> List.reduce add // Result: 1
twoInts |> List.reduce add // Result: 3
zeroInts |> List.fold add 0 // No error, result: 0
oneInt |> List.fold add 0 // Result: 1
twoInts |> List.fold add 0 // Result: 3
See also Difference between fold and reduce? for more explanations.
(See edit at the bottom of this post)
I'm making a program in Elixir that counts the types of HTML tags from a list of tags that I've already obtained. This means that the key should be the tag and the value should be the count.
e.g. in the following sample file
<html><head><body><sometag><sometag><sometag2><sometag>
My output should be something like the following:
html: 1
head: 1
body: 1
sometag: 3
sometag2: 1
Here is my code:
def tags(page) do
taglist = Regex.scan(~r/<[a-zA-Z0-9]+/, page)
dict = Map.new()
Enum.map(taglist, fn(x) ->
tag = String.to_atom(hd(x))
Map.put_new(dict, tag, 1)
end)
end
I know I should be probably using Enum.each instead but when I do that my dictionary ends up just being empty instead of incorrect.
With Enum.map, this is the output I receive:
iex(15)> A3.test
[%{"<html" => 1}, %{"<body" => 1}, %{"<p" => 1}, %{"<a" => 1}, %{"<p" => 1},
%{"<a" => 1}, %{"<p" => 1}, %{"<a" => 1}, %{"<p" => 1}, %{"<a" => 1}]
As you can see, there are duplicate entries and it's turned into a list of dictionaries. For now I'm not even trying to get the count working, so long as the dictionary doesn't duplicate entries (which is why the value is always just "1").
Thanks for any help.
EDIT: ------------------
Okay so I figured out that I need to use Enum.reduce
The following code produces the output I'm looking for (for now):
def tags(page) do
rawTagList = Regex.scan(~r/<[a-zA-Z0-9]+/, page)
tagList = Enum.map(rawTagList, fn(tag) -> String.to_atom(hd(tag)) end)
Enum.reduce(tagList, %{}, fn(tag, acc) ->
Map.put_new(acc, tag, 1)
end)
end
Output:
%{"<a": 1, "<body": 1, "<html": 1, "<p": 1}
Now I have to complete the challenge of actually counting the tags as I go...If anyone can offer any insight on that I'd be grateful!
First of all, it is not the best idea to parse html with regexes. See this question for more details (especially the accepted answer).
Secondly, you are trying to write imperative code in functional language (this is about first version of your code). Variables in Elixir are immutable. dict will always be an empty map. Enum.map takes a list and always returns new list of the same length with all elements transformed. Your transformation function takes an empty map and puts one key-value pair into it.
As a result you get a list with one element maps. The line:
Map.put_new(dict, tag, 1)
doesn't update dict in place, but creates new one using old one, which is empty. In your example it is exactly the same as:
%{tag => 1}
You have couple of options to do it differently. Closest approach would be to use Enum.reduce. It takes a list, an initial accumulator and a function elem, acc -> new_acc.
taglist
|> Enum.reduce(%{}, fn(tag, acc) -> Map.update(acc, tag, 1, &(&1 + 1)) end)
It looks a little bit complicated, because there are couple of nice syntactic sugars. taglist |> Enum.reduce(%{}, fun) is the same as Enum.reduce(taglist, %{}, fun). &(&1 + 1) is shorthand for fn(counter) -> counter + 1 end.
Map.update takes four arguments: a map to update, key to update, initial value if key doesn't exist and a function that does something with the key if it exists.
So, those two lines of code do this:
iterate over list Enum.reduce
starting with empty map %{}
take current element and map fn(tag, acc) and either:
if key doesn't exist insert 1
if it exists increment it by one &(&1 + 1)
Ok, I can't figure this one out even though I have an idea what it's doing...
let t = ["APE", "MONKEY", "DONKEY"]
Now consider three cases:
map (length.group) t
(map length.group) t
map (map length.group) t
This returns these three answers:
[3,6,6]
[1,1,1]
[[1,1,1],[1,1,1,1,1,1],[1,1,1,1,1,1]]
Now, can someone explain to me in details what's going on. A crucial part of this question is that I assume that map needs a list to work on and I don't see two maps being passed in the third case for example.
map (length.group) t
This composes the functions length and group. The result is a function that takes a list (string) and returns the number of "groups" in that list (where a group is a sequence of the same character repeating 1 or more times, so "abc" contains 3 groups and so does "aabbcc").
This function is then applied to each string in t using map.
(map length.group) t
Here the function map length (which takes the length of each sublist in a list of lists) is composed with the function group and the composed function is applied to t. In other words it's the same as map length (group t).
map (map length.group) t
Here the function map length . group is applied to each string in t, i.e. map length (group str) is calculated for each string str in t.
Try removing the "length." from all your cases, and see if that helps answer your question. It'll simplify the problem and the answer might show you a little better what's going on.
Or, factoring the third one, it becomes
map (map length.group) ["APE", "MONKEY", "DONKEY"]
--make parse order explicit
map ((map length) . group) ["APE", "MONKEY", "DONKEY"]
--do mapping
[((map length) . group) "APE", ((map length) . group) "MONKEY", ((map length) . group) "DONKEY"]
--use (f.g) x == f (g x)
[(map length) (group "APE"), ...]
[(map length) ["A", "P", "E"], ...]
[[1, 1, 1], ...]
Also try using some animals like "EEL" or "BEE" or "LLAMA" to see anything other than 1's in the final result.
I'm writing an R function, that is becoming quite big. It admit multiple choice, and I'm organizing it like so:
myfun <- function(y, type=c("aa", "bb", "cc", "dd" ... "zz")){
if (type == "aa") {
do something
- a lot of code here -
....
}
if (type == "bb") {
do something
- a lot of code here -
....
}
....
}
I have two questions:
Is there a better way, in order to not use the 'if' statement, for every choice of the parameter type?
Could it be more functional to write a sub-function for every "type" choice?
If I write subfunction, it would look like this:
myfun <- function(y, type=c("aa", "bb", "cc", "dd" ... "zz")){
if (type == "aa") result <- sub_fun_aa(y)
if (type == "bb") result <- sub_fun_bb(y)
if (type == "cc") result <- sub_fun_cc(y)
if (type == "dd") result <- sub_fun_dd(y)
....
}
Subfunction are of course defined elsewhere (in the top of myfun, or in another way).
I hope I was clear with my question. Thanks in Advance.
- Additional info -
I'm writing a function that applies some different filters to an image (different filter = different "type" parameter). Some filters share some code (for example, "aa" and "bb" are two gaussian filters, which differs only for one line code), while others are completely different.
So I'm forced to use a lot of if statement, i.e.
if(type == "aa" | type == "bb"){
- do something common to aa and bb -
if(type == "aa"){
- do something aa-related -
}
if(type == "bb"){
- do something bb-related -
}
}
if(type == "cc" | type == "dd"){
- do something common to cc and dd -
if(type == "cc"){
- do something cc-related -
}
if(type == "dd"){
- do something dd-related -
}
}
if(type == "zz"){
- do something zz-related -
}
And so on.
Furthermore, there are some if statement in the code "do something".
I'm looking for the best way to organize my code.
Option 1
One option is to use switch instead of multiple if statements:
myfun <- function(y, type=c("aa", "bb", "cc", "dd" ... "zz")){
switch(type,
"aa" = sub_fun_aa(y),
"bb" = sub_fun_bb(y),
"bb" = sub_fun_cc(y),
"dd" = sub_fun_dd(y)
)
}
Option 2
In your edited question you gave far more specific information. Here is a general design pattern that you might want to consider. The key element in this pattern is that there is not a single if in sight. I replace it with match.function, where the key idea is that the type in your function is itself a function (yes, since R supports functional programming, this is allowed).:
sharpening <- function(x){
paste(x, "General sharpening", sep=" - ")
}
unsharpMask <- function(x){
y <- sharpening(x)
#... Some specific stuff here...
paste(y, "Unsharp mask", sep=" - ")
}
hiPass <- function(x) {
y <- sharpening(x)
#... Some specific stuff here...
paste(y, "Hipass filter", sep=" - ")
}
generalMethod <- function(x, type=c(hiPass, unsharpMask, ...)){
match.fun(type)(x)
}
And call it like this:
> generalMethod("stuff", "unsharpMask")
[1] "stuff - General sharpening - Unsharp mask"
> hiPass("mystuff")
[1] "mystuff - General sharpening - Hipass filter"
There is hardly ever a reason not to refactor your code into smaller functions. In this case, besides the reorganisation, there is an extra advantage: the educated user of your function(s) can immediately call the subfunction if she knows where she's at.
If these functions have lots of parameters, a solution (to ease maintenance) could be to group them in a list of class "myFunctionParameters", but depends on your situation.
If code is shared between the different sub_fun_xxs, just plug that into another function that you use from within each of the sub_fun_xxs, or (if that's viable) calculate the stuff up front and pass it directly into each sub_fun_xx.
This is a much more general question about program design. There's no definitive answer, but there's almost certainly a better route than what you're currently doing.
Writing functions that handle the different types is a good route to go down. How effective it will be depends on several things - for example, how many different types are there? Are they at all related, e.g. could some of them be handled by the same function, with slightly different behavior depending on the input?
You should try to think about your code in a modular way. You have one big task to do overall. Can you break it down into a sequence of smaller tasks, and write functions that perform the smaller tasks? Can you generalize any of those tasks in a way that doesn't make the functions (much) more difficult to write, but does give them wider applicability?
If you give some more detail about what your program is supposed to be achieving, we will be able to help you more.
This is more of a general programming question than an R question. As such, you can follow basic guidelines of code quality. There are tools that can generate code quality reports from reading your code and give you guidelines on how to improve. One such example is Gendarme for .NET code. Here is a typical guideline that would appear in a report with too long methods:
AvoidLongMethodsRule