Removing Duplicates from a Matrix - octave

Let's say I have a matrix, where the values in in the first column are likely to repeat. Such as
1 2
3 4
5 6
1 3
Etc...
I want to get a matrix where the duplicates are added together into one row, with the second column being the sum of the entires in the duplicate row. For example the matrix above should be
1 2+3
3 4
5 6
Etc...
Any idea how to accomplish this? Would it be better to try to edit in place or copy the values into a new matrix?

The solution to your problem is to use accumarray. This, accumulating values in a new array, is the use case for the function. It will be much faster than a for loop and more readable (well, this is arguable. Many people also find it difficult but once you understand it, it's dead easy and very powerful):
octave> x = [1 2; 3 4; 5 6; 1 3];
octave> [y, ~, j] = unique (x(:,1));
octave> [y accumarray(j, x(:,2))]
ans =
1 5
3 4
5 6
Note that the above will work even when your data is not integers. This is because we are using the last output of unique(), j, which are indices into the array of unique values y.
octave> x = [1.1 2; 1.3 4; 1.5 6; 1.1 3];
octave> [y, i, j] = unique (x(:,1));
octave> [y accumarray(j, x(:,2))]
ans =
1.1000 5.0000
1.3000 4.0000
1.5000 6.0000

If the order of the rows of the summed matrix don't matter and you only have integer values, you could use unique to check each individual value in your first column. Like this:
m = [1 2; 3 4; 5 6; 1 3]; %original matrix
mvals = unique(m(:,1)); %unique of first column
mnew = zeros(length(mvals),size(m,2)); %preallocation
for k=1:length(mvals)
mnew(k,:) = [mvals(k), sum(m(m(:,1)==mvals(k),2:end),1)]; %sum each row where first is mvals(k)
end

Related

Selecting part of a matrix in Octave to form another matrix

Suppose I have the matrix
a = [1 2 3;
4 5 6;
7 8 9;]
I want to select the first two columns to form a matrix
b = [1 2;
4 5;
7 8;]
How to achieve this in Octave?
I know how to select a single column, but how to select many columns (let's say, the first 8 columns of a matrix having 16 columns) and form a matrix with them?
Also, how to select rows in a similar manner to form a matrix?
You can use the following code
b = a(:,1:2)
where : means taking all rows, and 1:2 means taking columns from 1 to 2.

How to prevent Octave from summing row and column vectors?

In Octave, in general, '+' will only work when the two operands have the same dimension.
There seems to be an exception to this rule: if you '+' a row vector (1 x n) and a column vector (n x 1), Octave will produce a (reasonable) Matrix of dimensions (n x n):
>> a = [1, 2, 3, 4, 5]
a =
1 2 3 4 5
>> b = [1; 2; 3; 4; 5]
b =
1
2
3
4
5
>> a+b
ans =
2 3 4 5 6
3 4 5 6 7
4 5 6 7 8
5 6 7 8 9
6 7 8 9 10
Can this be prevented, or at least be made to generate a warning? It seems potentially unsafe; I was recently bitten by a bug that was being masked by this behaviour.
Thanks!
No, this cannot be prevented. You need to check the input of your functions. The exception you mention is not an exception, it's the language syntax.
Element-wise operations require that dimensions have dimensions of equal lengths or length of one. The feature you are trying to prevent is also the feature that makes this work:
octave:1> a = 1:4
a =
1 2 3 4
octave:2> a+1
ans =
2 3 4 5
octave:3> a == 2
ans =
0 1 0 0
In the above examples, the value in the dimension with length 1 (1x1) is broadcasted or expanded. This feature is named Broadcasting in Octave and Python, and Implicit Expansion in Matlab. There's a bunch of operators and functions, such as == and max which also broadcast.
For a while, in Octave 3.6 and 3.8, it was possible to disable this by turning the Octave:broadcast into an error. However, because the way errors are handled in the language, that effectively made all Octave functions that used broadcasting to error.

KDB: apply dyadic function across two lists

Consider a function F[x;y] that generates a table. I also have two lists; xList:[x1;x2;x3] and yList:[y1;y2;y3]. What is the best way to do a simple comma join of F[x1;y1],F[x1;y2],F[x1;y3],F[x2;y1],..., thereby producing one large table?
You have asked for the cross product of your argument lists, so the correct answer is
raze F ./: xList cross yList
Depending on what you are doing, you might want to look into having your function operate on the entire list of x and the entire list of y and return a table, rather than on each pair and then return a list of tables which has to get razed. The performance impact can be considerable, for example see below
q)g:{x?y} //your core operation
q)//this takes each pair of x,y, performs an operation and returns a table for each
q)//which must then be flattened with raze
q)fm:{flip `x`y`res!(x;y; enlist g[x;y])}
q)//this takes all x, y at once and returns one table
q)f:{flip `x`y`res!(x;y;g'[x;y])}
q)//let's set a seed to compare answers
q)\S 1
q)\ts do[10000;rm:raze fm'[x;y]]
76 2400j
q)\S 1
q)\ts do[10000;r:f[x;y]]
22 2176j
q)rm~r
1b
Setup our example
q)f:{([] total:enlist x+y; x:enlist x; y:enlist y)}
q)x:1 2 3
q)y:4 5 6
Demonstrate F[x1;y1]
q)f[1;4]
total x y
---------
5 1 4
q)f[2;5]
total x y
---------
7 2 5
Use the multi-valent apply operator together with each' to apply to each pair of arguments.
q)raze .'[f;flip (x;y)]
total x y
---------
5 1 4
7 2 5
9 3 6
Another way to achieve it using each-both :
x: 1 2 3
y: 4 5 6
f:{x+y}
f2:{ a:flip x cross y ; f'[a 0;a 1] }
f2[x;y]
5j, 6j, 7j, 6j, 7j, 8j, 7j, 8j, 9j

GNU Octave: How to find n maximum elements of a vector

I have vector in Octave like this:
[ 4 5 1 2 3 6 ]
Is there any function that returns n maximum elements of that vector, in this case, the three biggest are 6, 5, and 4?
[6 5 4]
The Octave max function only returns one maximum element. I want n maximum elements.
In GNU Octave, get the biggest n elements of a vector:
octave:2> X = [3 8 2 9 4]
octave:2> sort(X)
ans =
2 3 4 8 9
octave:8> sort(X)(end-2:end)
ans =
4 8 9
Description
What sort(X)(end-2:end) means is "sort the vector X, and give me the elements from 2 minus the end to the end, also known as the last 3 elements".
You can use the sort function for this.

How Does Modulus Divison Work

I don't really understand how modulus division works.
I was calculating 27 % 16 and wound up with 11 and I don't understand why.
I can't seem to find an explanation in layman's terms online.
Can someone elaborate on a very high level as to what's going on here?
Most explanations miss one important step, let's fill the gap using another example.
Given the following:
Dividend: 16
Divisor: 6
The modulus function looks like this:
16 % 6 = 4
Let's determine why this is.
First, perform integer division, which is similar to normal division, except any fractional number (a.k.a. remainder) is discarded:
16 / 6 = 2
Then, multiply the result of the above division (2) with our divisor (6):
2 * 6 = 12
Finally, subtract the result of the above multiplication (12) from our dividend (16):
16 - 12 = 4
The result of this subtraction, 4, the remainder, is the same result of our modulus above!
The result of a modulo division is the remainder of an integer division of the given numbers.
That means:
27 / 16 = 1, remainder 11
=> 27 mod 16 = 11
Other examples:
30 / 3 = 10, remainder 0
=> 30 mod 3 = 0
35 / 3 = 11, remainder 2
=> 35 mod 3 = 2
The simple formula for calculating modulus is :-
[Dividend-{(Dividend/Divisor)*Divisor}]
So, 27 % 16 :-
27- {(27/16)*16}
27-{1*16}
Answer= 11
Note:
All calculations are with integers. In case of a decimal quotient, the part after the decimal is to be ignored/truncated.
eg: 27/16= 1.6875 is to be taken as just 1 in the above mentioned formula. 0.6875 is ignored.
Compilers of computer languages treat an integer with decimal part the same way (by truncating after the decimal) as well
Maybe the example with an clock could help you understand the modulo.
A familiar use of modular arithmetic is its use in the 12-hour clock, in which the day is divided into two 12 hour periods.
Lets say we have currently this time: 15:00
But you could also say it is 3 pm
This is exactly what modulo does:
15 / 12 = 1, remainder 3
You find this example better explained on wikipedia: Wikipedia Modulo Article
The modulus operator takes a division statement and returns whatever is left over from that calculation, the "remaining" data, so to speak, such as 13 / 5 = 2. Which means, there is 3 left over, or remaining from that calculation. Why? because 2 * 5 = 10. Thus, 13 - 10 = 3.
The modulus operator does all that calculation for you, 13 % 5 = 3.
modulus division is simply this : divide two numbers and return the remainder only
27 / 16 = 1 with 11 left over, therefore 27 % 16 = 11
ditto 43 / 16 = 2 with 11 left over so 43 % 16 = 11 too
Very simple: a % b is defined as the remainder of the division of a by b.
See the wikipedia article for more examples.
I would like to add one more thing:
it's easy to calculate modulo when dividend is greater/larger than divisor
dividend = 5
divisor = 3
5 % 3 = 2
3)5(1
3
-----
2
but what if divisor is smaller than dividend
dividend = 3
divisor = 5
3 % 5 = 3 ?? how
This is because, since 5 cannot divide 3 directly, modulo will be what dividend is
I hope these simple steps will help:
20 % 3 = 2
20 / 3 = 6; do not include the .6667 – just ignore it
3 * 6 = 18
20 - 18 = 2, which is the remainder of the modulo
Easier when your number after the decimal (0.xxx) is short. Then all you need to do is multiply that number with the number after the division.
Ex: 32 % 12 = 8
You do 32/12=2.666666667
Then you throw the 2 away, and focus on the 0.666666667
0.666666667*12=8 <-- That's your answer.
(again, only easy when the number after the decimal is short)
27 % 16 = 11
You can interpret it this way:
16 goes 1 time into 27 before passing it.
16 * 2 = 32.
So you could say that 16 goes one time in 27 with a remainder of 11.
In fact,
16 + 11 = 27
An other exemple:
20 % 3 = 2
Well 3 goes 6 times into 20 before passing it.
3 * 6 = 18
To add-up to 20 we need 2 so the remainder of the modulus expression is 2.
The only important thing to understand is that modulus (denoted here by % like in C) is defined through the Euclidean division.
For any two (d, q) integers the following is always true:
d = ( d / q ) * q + ( d % q )
As you can see the value of d%q depends on the value of d/q. Generally for positive integers d/q is truncated toward zero, for instance 5/2 gives 2, hence:
5 = (5/2)*2 + (5%2) => 5 = 2*2 + (5%2) => 5%2 = 1
However for negative integers the situation is less clear and depends on the language and/or the standard. For instance -5/2 can return -2 (truncated toward zero as before) but can also returns -3 (with another language).
In the first case:
-5 = (-5/2)*2 + (-5%2) => -5 = -2*2 + (-5%2) => -5%2 = -1
but in the second one:
-5 = (-5/2)*2 + (-5%2) => -5 = -3*2 + (-5%2) => -5%2 = +1
As said before, just remember the invariant, which is the Euclidean division.
Further details:
What is the behavior of integer division?
Division and Modulus for Computer Scientists
Modulus division gives you the remainder of a division, rather than the quotient.
It's simple, Modulus operator(%) returns remainder after integer division. Let's take the example of your question. How 27 % 16 = 11? When you simply divide 27 by 16 i.e (27/16) then you get remainder as 11, and that is why your answer is 11.
Lets say you have 17 mod 6.
what total of 6 will get you the closest to 17, it will be 12 because if you go over 12 you will have 18 which is more that the question of 17 mod 6. You will then take 12 and minus from 17 which will give you your answer, in this case 5.
17 mod 6=5
Modulus division is pretty simple. It uses the remainder instead of the quotient.
1.0833... <-- Quotient
__
12|13
12
1 <-- Remainder
1.00 <-- Remainder can be used to find decimal values
.96
.040
.036
.0040 <-- remainder of 4 starts repeating here, so the quotient is 1.083333...
13/12 = 1R1, ergo 13%12 = 1.
It helps to think of modulus as a "cycle".
In other words, for the expression n % 12, the result will always be < 12.
That means the sequence for the set 0..100 for n % 12 is:
{0,1,2,3,4,5,6,7,8,9,10,11,0,1,2,3,4,5,6,7,8,9,10,11,0,[...],4}
In that light, the modulus, as well as its uses, becomes much clearer.
Write out a table starting with 0.
{0,1,2,3,4}
Continue the table in rows.
{0,1,2,3,4}
{5,6,7,8,9}
{10,11,12,13,14}
Everything in column one is a multiple of 5. Everything in column 2 is a
multiple of 5 with 1 as a remainder. Now the abstract part: You can write
that (1) as 1/5 or as a decimal expansion. The modulus operator returns only
the column, or in another way of thinking, it returns the remainder on long
division. You are dealing in modulo(5). Different modulus, different table.
Think of a Hash Table.
When we divide two integers we will have an equation that looks like the following:
A/B​​ =Q remainder R
A is the dividend; B is the divisor; Q is the quotient and R is the remainder
Sometimes, we are only interested in what the remainder is when we divide A by B.
For these cases there is an operator called the modulo operator (abbreviated as mod).
Examples
16/5= 3 Remainder 1 i.e 16 Mod 5 is 1.
0/5= 0 Remainder 0 i.e 0 Mod 5 is 0.
-14/5= 3 Remainder 1 i.e. -14 Mod 5 is 1.
See Khan Academy Article for more information.
In Computer science, Hash table uses Mod operator to store the element where A will be the values after hashing, B will be the table size and R is the number of slots or key where element is inserted.
See How does a hash table works for more information
This was the best approach for me for understanding modulus operator. I will just explain to you through examples.
16 % 3
When you division these two number, remainder is the result. This is the way how i do it.
16 % 3 = 3 + 3 = 6; 6 + 3 = 9; 9 + 3 = 12; 12 + 3 = 15
So what is left to 16 is 1
16 % 3 = 1
Here is one more example: 16 % 7 = 7 + 7 = 14 what is left to 16? Is 2 16 % 7 = 2
One more: 24 % 6 = 6 + 6 = 12; 12 + 6 = 18; 18 + 6 = 24. So remainder is zero, 24 % 6 = 0