Let's say I have a sequence of values (e.g., 3, 5, 8, 12, 15) and I want to occasionally decrease all of them by a certain value.
If I store them as the sequence (0, 2, 3, 4, 3) and keep a variable as a base of 3, I now only have to change the base (and check the first items) whenever I want to decrease them instead of actually going over all the values.
I know there's an official term for this, but when I literally translate from my native language to English it doesn't come out right.
Differential Coding / Delta Encoding?
I don't know a name for the data structure, but it's basically just base+offset :-)
An offset?
If I understand your question right, you're rebasing. That's normally used in reference to patching up addresses in DLLs from a load address.
I'm not sure that's what you're doing, because your example seems to be incorrect. In order to come out with { 3, 5, 8, 12, 15 }, with a base of 3, you'd need { 0, 2, 5, 9, 12 }.
I'm not sure. If you imagine your first array as providing the results of some function of an index value f(i) where f(0) is 3, f(1) is 5, and so forth, then your second array is describing the function f`(i) where f(i+1) = f(i) + f'(i) given f(0) = 3.
I'd call it something like a derivative function, where the process of retrieving your original data is simply the summation function.
What will happen more often, will you be changing f(0) or retrieving values from f(i)? Is this technique rooted in a desire to optimize?
Perhaps you're looking for a term like "Inductive Sequence" or "Induction Sequence." (I just made that up.)
Related
I have an RL problem where I want the agent to make a selection of x out of an array of size n.
I.e. if I have [0, 1, 2, 3, 4, 5] then n = 6 and if x = 3 a valid action could be
[2, 3, 5].
Right now what I tried is have n scores:
Output n continuous numbers, and select the x highest ones. This works quite ok.
And I tried iteratively replacing duplicates out of a Multi Discrete action. Where we have x values that can be anything from 0 to n-1.
Is there some other optimal action space I am missing that would force the agent to make unique choices?
Many thanks for your valuable insights and tips in advance! I am happy to try all!
Since reinforcement learning mostly about interacting with environment, you can approach like this:
Your agent starts choosing actions. After choosing the first action, you can either update the possible choices it has by removing the last choice (with temporary action list) or you can update the values of the chosen action (giving it either negative reward or punishing it). I think this could solve your problem.
I am experimenting with reinforcement learning in python using Keras. Most of the tutorials available use OpenAI Gym library to create the environment, state, and action sets.
After practicing with many good examples written by others, I decided that I want to create my own reinforcement learning environment, state, and action sets.
This is what I think will be fun to teach the machine to do.
An array of integers from 1 to 4. I will call these targets.
targets = [[1, 2, 3, 4]]
Additional numbers list (at random) from 1 to 4. I will call these bullets.
bullets = [1, 2, 3, 4]
When I shoot a bullet to a target, the target's number will be the sum of original target num + bullet num.
I want to shoot a bullet (one at a time) at one of the targets to make
For example, given targets [1 2 3 4] and bullet 1, I want the machine to predict the correct index to shoot at.
In this case, it should be index 3, because 4 + 1 = 5
curr_state = [[1, 2, 3, 4]]
bullet = 1
action = 3 (<-- index of the curr_state)
next_state = [[1, 2, 3, 5]]
I have been picking my brain to think of the best way to construct this into a reinforcement design. I tried some, but the model result is not very good (meaning, it most likely fails to make number 5).
Mostly because the state is a 2D: (1) targets; (2) bullet at that time. The method I employed so far is to convert the state as the following:
State = 5 - targets - bullet
I was wondering if anyone can think of a better way to design this model?
Thanks in advance!
Alright, so it looks like no one is helping you out, so I just wrote a Python environment file for you as you described. I also made it as much OpenAI style for you as possible, here is the link to it, it is in my GitHub repository. You can copy the code or fork it. I will explain it below:
https://github.com/RuiNian7319/Miscellaneous/blob/master/ShootingRange.py
States = [0, 1, 2, ..., 10]
Actions = [-2, -1, 0, 1, 2]
So the game starts at a random number between 0 - 10 (you can change this easily if you want), and the random number is your "target" you described above. Given this target, your AI agent can fire the gun, and it shoots bullets corresponding to the numbers above. The objective is for your bullet and the target to add up to 5. There are negatives in case your AI agent overshoots 5, or if the target is a number above 5.
To get a positive reward, the agent has to get 5. So if the current value is 3, and the agent shoots 2, then the agent will get a reward of 1 since he got the total value of 5, and that episode will end.
There are 3 ways for the game to end:
1) Agent gets 5
2) Agent fails to get 5 in 15 tries
3) The number is above 10. In this case, we say the target is too far
Sometimes, you need to shoot multiple times to get 5. So, if your agent shoots, its current bullet will be added to the state, and the agent tries again from that new state.
Example:
Current state = 2. Agent shoots 2. New state is 4. And the agent starts at 4 at the next time step. This "sequential decision making" creates a reinforcement learning environment, rather than a contextual bandit.
I hope this makes sense, let me know if you have any questions.
(This might be a bizarre question, so please don't tell me "why do you want to do this?".)
I'm wondering whether there are methods or tricks or tools or... to write a function using another function (the specific language doesn't really matter). For example, assume that you are given a function "power" defined as follows:
int power (int n) { return n == 0 ? 1 : 2 * power (n - 1); }
and assume that you are asked to write, using that function or (even better) by transforming that function, another one that computes for example the Lucas numbers (2, 1, 3, 4, 7, 11, 18, 29, 47, 76, 123, 199, 322, 521, ...). Of course the trivial way to use the given function would be to do:
int lucas (int n) { ... + power (n) - power (n) + ... }
but this is not the kind of answer I'm looking for. Likewise the trivial way to transform the function is to delete its body and writing another one instead, but again it's obviously not what I want to do.
Yes.
That is pretty standard in many languages.
Functions take some sort of input parameters and may call other functions or itself, and eventually return some sort of value.
If you have a specific language in mind, someone might show you a
straightforward example.
I'm just learning about CUDA/OpenCL, but I have a conceptual question. Suppose I am encoding an algorithm to do a Breadth-First-Search on a graph. Suppose my target device is a GPU with only 2 work-groups of 2 processing elements (i.e., 4 cores). Intuitivelly, I guess the search could be done in parallel by keeping an array of "nodes to visit". For each pass, each node is visited in parallel and added to the next "nodes to visit" array. For example, this graph could spawn the following search:
start: must visit A
parallel pass 1: must visit B, C, D
parallel pass 2: must visit E, F, G, H, I, J, K, L, M
parallel pass 3: must visit N, O, P
parallel pass 4: must visit Q
parallel pass 5: done
A way to do it, I suppose, would be to call a ND range kernel 5 times (example in OpenCL):
clEnqueueNDRangeKernel(queue, kernel, 1, 0, 1, 1, 0, NULL, NULL);
clEnqueueNDRangeKernel(queue, kernel, 1, 0, 1, 3, 0, NULL, NULL);
clEnqueueNDRangeKernel(queue, kernel, 1, 0, 1, 9, 0, NULL, NULL);
clEnqueueNDRangeKernel(queue, kernel, 1, 0, 1, 3, 0, NULL, NULL);
clEnqueueNDRangeKernel(queue, kernel, 1, 0, 1, 1, 0, NULL, NULL);
(Of course, this is hard-coded for this case, so, to avoid that, I guess I could keep a counter of nodes to visit.) This sounds wrong, though: it doesn't fit the layout of a GPU, at all. For example, on the third call, you are using 1 work group of 9 work items, when your GPU has 2 work groups of 2 work items...!
Another way I see could be the following (example in OpenCL):
while (work not complete...)
clEnqueueNDRangeKernel(queue, kernel, 1, 0, 2, 2, 0, NULL, NULL);
This would call "clEnqueueNDRangeKernel" continously in a way that fits perfectly the GPU layout, until it receives a signal that the work is done. But this time, the ids received by the kernel wouldn't fit the layout of the algorithm!
What is the right way to do this? Am I missing something?
Your question is one of the most unobvious & interesting.
IMO, you should implement "pure" & easy-to-analyze algorithm if want your application to run on different hardware. If someone else will come across your implementation, at least, it will be easy to tweak.
Otherwise, if you put performance first, hardcode every single piece of software to achieve optimal performance on single target platform. Other man, who may work with your code later, will have to learn hardware peculiarities anyway.
From what I can infer from your question, I'd say that the short answer is both.
I say that because the way you want to solve a problem is always linked to how you posed that problem.
The best illustration is the amount of sorting algorithms that exist. From the pure sorting point of view ("I need my data sorted"), the problem is solved for a while. However it didn't stop researchers to investigate new algorithms because they added constraints and/or new input to the problem: "I need my data sorted as fast as possible" (the reason why algorithms are categorized with the big O notation), "I need my data sorted as fast as possible knowing that the data structure is..." or "knowing that there is X % chance that..." or "I don't care about the speed but I care about memory", etc.
Now your problem seems to be: I want a breadth first search algorithm that runs efficiently (this I'm guessing - why to learn OCL/CUDA otherwise?) on GPUs.
This simple sentence hides a lot of constraints. For instance you have to take into account that:
it takes a lot of time to send the data through the PCIe bus (for discrete GPUs).
Access (global) memory latency is high.
Threads work in lock steps (number varies with vendors).
Latency is hidden with throughput, and so on.
Note also that it is not necessarily the same that "I want a parallel breadth first search algorithm" (which could run on CPU with again different constraints).
A quick Google search with these key words : "breadth first search parallel GPU" returns me, among others these articles that seems promising (I just went through the abstracts):
Efficient Parallel Graph Exploration on Multi-Core CPU and GPU (Use CPU and GPU)
An Effective GPU Implementation of Breadth-First Search
Scalable GPU Graph Traversal (From NVIDIA)
Here is a function I would like to write but am unable to do so. Even if you
don't / can't give a solution I would be grateful for tips. For example,
I know that there is a correlation between the ordered represantions of the
sum of an integer and ordered set partitions but that alone does not help me in
finding the solution. So here is the description of the function I need:
The Task
Create an efficient* function
List<int[]> createOrderedPartitions(int n_1, int n_2,..., int n_k)
that returns a list of arrays of all set partions of the set
{0,...,n_1+n_2+...+n_k-1} in number of arguments blocks of size (in this
order) n_1,n_2,...,n_k (e.g. n_1=2, n_2=1, n_3=1 -> ({0,1},{3},{2}),...).
Here is a usage example:
int[] partition = createOrderedPartitions(2,1,1).get(0);
partition[0]; // -> 0
partition[1]; // -> 1
partition[2]; // -> 3
partition[3]; // -> 2
Note that the number of elements in the list is
(n_1+n_2+...+n_n choose n_1) * (n_2+n_3+...+n_n choose n_2) * ... *
(n_k choose n_k). Also, createOrderedPartitions(1,1,1) would create the
permutations of {0,1,2} and thus there would be 3! = 6 elements in the
list.
* by efficient I mean that you should not initially create a bigger list
like all partitions and then filter out results. You should do it directly.
Extra Requirements
If an argument is 0 treat it as if it was not there, e.g.
createOrderedPartitions(2,0,1,1) should yield the same result as
createOrderedPartitions(2,1,1). But at least one argument must not be 0.
Of course all arguments must be >= 0.
Remarks
The provided pseudo code is quasi Java but the language of the solution
doesn't matter. In fact, as long as the solution is fairly general and can
be reproduced in other languages it is ideal.
Actually, even better would be a return type of List<Tuple<Set>> (e.g. when
creating such a function in Python). However, then the arguments wich have
a value of 0 must not be ignored. createOrderedPartitions(2,0,2) would then
create
[({0,1},{},{2,3}),({0,2},{},{1,3}),({0,3},{},{1,2}),({1,2},{},{0,3}),...]
Background
I need this function to make my mastermind-variation bot more efficient and
most of all the code more "beautiful". Take a look at the filterCandidates
function in my source code. There are unnecessary
/ duplicate queries because I'm simply using permutations instead of
specifically ordered partitions. Also, I'm just interested in how to write
this function.
My ideas for (ugly) "solutions"
Create the powerset of {0,...,n_1+...+n_k}, filter out the subsets of size
n_1, n_2 etc. and create the cartesian product of the n subsets. However
this won't actually work because there would be duplicates, e.g.
({1,2},{1})...
First choose n_1 of x = {0,...,n_1+n_2+...+n_n-1} and put them in the
first set. Then choose n_2 of x without the n_1 chosen elements
beforehand and so on. You then get for example ({0,2},{},{1,3},{4}). Of
course, every possible combination must be created so ({0,4},{},{1,3},{2}),
too, and so on. Seems rather hard to implement but might be possible.
Research
I guess this
goes in the direction I want however I don't see how I can utilize it for my
specific scenario.
http://rosettacode.org/wiki/Combinations
You know, it often helps to phrase your thoughts in order to come up with a solution. It seems that then the subconscious just starts working on the task and notifies you when it found the solution. So here is the solution to my problem in Python:
from itertools import combinations
def partitions(*args):
def helper(s, *args):
if not args: return [[]]
res = []
for c in combinations(s, args[0]):
s0 = [x for x in s if x not in c]
for r in helper(s0, *args[1:]):
res.append([c] + r)
return res
s = range(sum(args))
return helper(s, *args)
print partitions(2, 0, 2)
The output is:
[[(0, 1), (), (2, 3)], [(0, 2), (), (1, 3)], [(0, 3), (), (1, 2)], [(1, 2), (), (0, 3)], [(1, 3), (), (0, 2)], [(2, 3), (), (0, 1)]]
It is adequate for translating the algorithm to Lua/Java. It is basically the second idea I had.
The Algorithm
As I already mentionend in the question the basic idea is as follows:
First choose n_1 elements of the set s := {0,...,n_1+n_2+...+n_n-1} and put them in the
first set of the first tuple in the resulting list (e.g. [({0,1,2},... if the chosen elements are 0,1,2). Then choose n_2 elements of the set s_0 := s without the n_1 chosen elements beforehand and so on. One such a tuple might be ({0,2},{},{1,3},{4}). Of
course, every possible combination is created so ({0,4},{},{1,3},{2}) is another such tuple and so on.
The Realization
At first the set to work with is created (s = range(sum(args))). Then this set and the arguments are passed to the recursive helper function helper.
helper does one of the following things: If all the arguments are processed return "some kind of empty value" to stop the recursion. Otherwise iterate through all the combinations of the passed set s of the length args[0] (the first argument after s in helper). In each iteration create the set s0 := s without the elements in c (the elements in c are the chosen elements from s), which is then used for the recursive call of helper.
So what happens with the arguments in helper is that they are processed one by one. helper may first start with helper([0,1,2,3], 2, 1, 1) and in the next invocation it is for example helper([2,3], 1, 1) and then helper([3], 1) and lastly helper([]). Of course another "tree-path" would be helper([0,1,2,3], 2, 1, 1), helper([1,2], 1, 1), helper([2], 1), helper([]). All these "tree-paths" are created and thus the required solution is generated.