so my data structure class is covering time complexity and I just have a quick question about the performance for arraylist and treemap.
The get method for ArrayList is O(1) and the get method for TreeMap is o(log n)
Now, if i made a loop that iterates through the entire list or tree such as
for (int i = 0; i < blahblah.size(); i++)
{
blah blah
}
For arraylist, would this loop performance be o(1) or o(n)? I realize that when you are retrieving 1 item, the performance is O(1) but this loop goes through the entire list so wouldn't it be 1 * n items which would make it n?
Same thing with the treemap would it be o(log n) or n log n since you're going through the entire tree.
Since you want all n elements of the data structure, the result is not just dependent of the given data structure but also of the number of elements you want to retrieve.
So yes the result would be O(n) or O(n log n), but depending on the data structure you could do better. Consider a linked list - instead of getting O(n * n) you can do just fine with O(n).
But then there's more to performance than just big O - constants do matter in reality. One example would be radix sort of 32bit numbers. O(n) for sorting sounds great, but a quick sort will be most certainly still be faster for most reasonable input sizes.
"The get method for ArrayList is O(1) and the get method for TreeMap is o(log n)"
The reason ArrayList.get() is O(1) is because looking up by index is just an offset from origin. It doesn't matter if the array has 5 or 5M elements it's the same amount of work.
TreeMap.get() is O(log n) because it may have to traverse multiple elements downwards the binary tree.
Yes, that would be O(n). O values are almost always worst case.
Going through all elements would be O(n) since you'd have to visit each at least once.
The treemap should be O(n) as well for the same reason regardles of whether you visit the nodes in postorder, inorder or preorder order.
Related
Given 3 sorted lists L1,L2,L3 all of size n and a number x, whats the fastest algorithm that can return up to 3 numbers, at most 1 from each list, such that the sum of those returned numbers add to x.
The fastest algorithm I can think of, checks all possible combinations which is O(n^3). Is there a better way?
Thanks
You can get to O(n^2):
Consider only two lists, L1 and L2. Set an index in L1 at the maximum element, and an index in L2 at an added zero element. Now iterate both indexes together: If the sum of the selected two elements is too big, step the L1 index down. If the sum of the selected two elements is too small, step the L2 index up. This takes O(2n) time.
Essentially, you're constructing a rectangle, with the elements of L1 being one axis and the elements of L2 being the Y axis. Then you walk the boundary between the region of points representing sums that are too large and the region of points representing sums that are too small.
Executing the above algorithm once for each value in L3 gives an O(n^2) algorithm.
You can at least get to O(n^2 log n) because, given 2 indices (i and j, say), you can do a binary search in L3 for x-L1[i]-L2[j].
If you don't need a value from every list, you could try each possible non-empty subset of lists individually; since there 6 such subsets, and solving for each will be faster than O(n^2 log n) (O(n log n) for pairs of lists, O(log n) for individual lists), the overall complexity is unchanged.
I have a large rectangular matrix NxM in GPU memory, stored as 1-dimensional array in row-by-row representation. Let us say that this matrix is actually composed of submatrices of size nxm. For simplicity, assume that N is a multiple of n and same with M and m. Let us say, the data type of the array is float or double.
What is an efficient method to find the index of the extrema in each sub-matrix? For example, how to find the 1-dimensional index of the maximum element of each submatrix and write down those indices in some array.
I can hardly imagine to be so self-confident (or arrogant?) to say that one particular solution is the "most efficient way" to do something.
However, some thoughts (without the claim to cover the "most efficient" solution) :
I think that there are basically two "orthogonal" ways of approaching this
For all sub-matrices in parallel: Find the extremum sequentially
For all sub-matrices sequentially: Find the extremum in parallel
The question which one is more appropriate probably depends on the sizes of the matrices. You mentioned that "N is a multiple of n" (similarly for M and m). Let's the matrix of size M x N is composed of a*b sub-matrices of size m x n.
For the first approach, one could simply let each thread take care of one sub-matrix, with a trivial loop like
for (all elements of my sub-matrix) max = element > max ? element : max;
The prerequisite here is that a*b is "reasonably large". That is, when you can launch this kernel for, let's say, 10000 sub-matrices, then this could already bring a good speedup.
In contrast to that, in the second approach, each kernel (with all its threads) would take care of one sub-matrix. In this case, the kernel could be a standard "reduction" kernel. (The reduction is often presented an example for "computing the sum/product of the elements of an array", but it works for any binary associative operation, so instead of computing the sum or product, one can basically use the same kernel for computing the minimum or maximum). So the kernel would be launched for each sub-matrix, and this would only make sense when the sub-matrix is "reasonably large".
However, in both cases, one has to consider the general performance guidelines. Particularly, since in this case, the operation is obviously memory-bound (and not compute-bound), one has to make sure that the accesses to global memory (that is, to the matrix itself) are coalesced, and that the occupancy that is created by the kernel is as high as possible.
EDIT: Of course, one could consider to somehow combine these approaches, but I think that they are at least showing the most important directions of the space of available options.
Does there exist an algorithm which, given an ordered list of symbols {a1, a2, a3, ..., ak}, produces in O(n) time a new list of the same symbols in a random order without bias? "Without bias" means the probability that any symbol s will end up in some position p in the list is 1/k.
Assume it is possible to generate a non-biased integer from 1-k inclusive in O(1) time. Also assume that O(1) element access/mutation is possible, and that it is possible to create a new list of size k in O(k) time.
In particular, I would be interested in a 'generative' algorithm. That is, I would be interested in an algorithm that has O(1) initial overhead, and then produces a new element for each slot in the list, taking O(1) time per slot.
If no solution exists to the problem as described, I would still like to know about solutions that do not meet my constraints in one or more of the following ways (and/or in other ways if necessary):
the time complexity is worse than O(n).
the algorithm is biased with regards to the final positions of the symbols.
the algorithm is not generative.
I should add that this problem appears to be the same as the problem of randomly sorting the integers from 1-k, since we can sort the list of integers from 1-k and then for each integer i in the new list, we can produce the symbol ai.
Yes - the Knuth Shuffle.
The Fisher-Yates Shuffle (Knuth Shuffle) is what you are looking for.
I need a data structure that is ordered but also gives fast random access and inserts and removes. Linkedlists are ordered and fast in inserts and removes but they give slow random access. Hashtables give fast random access but are not ordered.
So, it seems to nice to use both of them together. In my current solution, my Hashtable includes iterators of the list and the List contains the actual items. Nice and effective. Okay, it requires double the memory but that's not an issue.
I have heard that some tree structures could do this also, but are they as fast as this solution?
The most efficient tree structure I know is Red Black Tree, and it's not as fast as your solution as it has O(log n) for all operations while your solution has O(1) for some, if not all, operations.
If memory is not an issue and you sure your solution is O(1) meaning the time required to add/delete/find item in the structure is not related to the amount of items you have, go for it.
You should consider a Skip List, which is an ordered linked-list with O(log n) access times. In other words, you can enumerate it O(n) and index/insert/delete is O(log n).
Trees are made for this. The most appropriate are self-balancing trees like AVL tree or Red Black tree. If you deal with a very big data amounts, it also may be useful to create B-tree (they are used for filesystems, for example).
Concerning your implementation: it may be more or less efficient then trees depending on data amount you work with and HashTable implementation. E.g. some hash tables with a very dense data may give access not in O(1) but in O(log n) or even O(n). Also remember that computing hash from data takes some time too, so for a quit small data amounts absolute time for computing hash may be more then for searching it in a tree.
What you did is pretty much the right choice.
The cool thing about this is that adding ordering to an existing map implementation by using a double-ended doubly-linked list doesn't actually change its asymptotic complexity, because all the relevant list operations (appending and deleting) have a worst-case step complexity of Θ(1). (Yes, deletion is Θ(1), too. The reason it is usually Θ(n) is because you have to find the element to delete first, which is Θ(n), but the actual deletion itself is Θ(1). In this particular case, you let the map do the finding, which is something like Θ(1) amortized worst-case step complexity or Θ(logb n) worst-case step complexity, depending on the type of map implementation used.)
The Hash class in Ruby 1.9, for example, is an ordered map, and it is implemented at least in YARV and Rubinius as a hash table embedded into a linked list.
Trees generally have a worst-case step complexity of Θ(logb n) for random access, whereas hash tables may be worse in the worst case (Θ(n)), but usually amortize to Θ(1), provided you don't screw up the hash function or the resize function.
[Note: I'm deliberately only talking about asymptotic behavior here, aka "infinitely large" collections. If your collections are small, then just choose the one with the smallest constant factors.]
Java actually contains a LinkedHashTable, which is similar to the data-structure you're describing. It can be surprisingly useful at times.
Tree structures could work as well, seeing they can perform random access (and most other operations) in (O log n) time. Not as fast as Hashtables (O 1), but still fast unless your database is very large.
The only real advantage of trees is that you don't need to decide on the capacity beforehand. Some HashTable implementations can grow their capacity as needed, but simply do so by copying all items into a new, larger hashtable when they've exceeded their capacity, which is very slow. (O n)
First of all I assume I've missed something major when thinking about this, but I still wanted to post about it to see if I really didn't miss anything, on with it...
I have a pretty write heavy binary tree (about 50/50 between writes and reads), and on the way home today I was thinking about ways to optimize this, especially make the writes faster - this is what I came up with.
Considering that the operation add(T, x) to add x to tree T first consists of find(T, x) to see if x already exists, and in that case it doesn't return the parent so we can add it instead of one of the parents empty leaves.
What if we add a hash table as an intermediate cache to the add operation, so when we call add(T, x) what really happens is that x is hashed and inserted into the hash map M. And that's it. The optimization takes place when we somewhere else asks to find(T, x), now when we search the tree we will come to a leaf node, since x isn't inserted the tree yet (it only exists in the hash map M), we hash x and compare it to the keys in M to see if it is supposed to be in the tree. If it's found in M then we add it to the tree and remove it from M.
This would eliminate the find(T, x) operation on add(T, x) and reduce it to add(M, x) which is O(1). And then (ab)-use the find(T, x) operation that is performed when we look up the node the first time to insert it.
Why not use a hashtable for everything and omit the binary tree entirely?
It all depends why you were using binary trees in the first place. If you chose binary trees to enhance sharing, you lose with the hashtable caches because hashtables are not shared.
The caches do not make comparing two maps any easier either.
EDIT:
If the operations that take advantage of the specificities of trees are rare (you mention taking advantage of the fact that RB trees are sorted) and if, on the other hand, you often look up a key that has recently been added, or replace the value of a key that has recently been added, a small-size cache implemented with another structure may make sense. You can also consider using a hashtable representation with the occasional conversion to a tree.
The additional complexity of this cache layer may mean that you do not gain any time in practice though, or not enough to repay the technical debt of having an ad-hoc data structure like this.
If your need is to have a structure that has O(1) inserts, and approximately O(n) amortized ordered iteration, I had the same problem:
Key-ordered dict in Python
The answer (keeping a hash and a partially sorted list, and using a partially-sorted-structure-friendly sort like TimSort) worked very well in practice in my case.