Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I have a project in a course at the university that requires various Matlab functions. My version of Matlab is supplied by my workplace, and it doesn't have some of the toolboxes that are required for the project. Is there some repository of such implementations anywhere? I couldn't find anything relevant when I googled.
While my question is general, I'm listing the functions that I need and can't find, since my question is also specific:
knnclassify - an implementation of K-nearest neighbors
svmclassify - an implementation of Support Vector Machines
svmtrain - part of the SVM implementation
mapstd - normalizes a matrix to have a mean of 0 and standard deviation of 1
An alternative I'm considering is working in Python with Numpy and Pylab. Are there toolboxes in Pylab that are equivalent to these Matlab functions?
The first place I would check is the MathWorks File Exchange. There are over 10,000 code submissions from MATLAB users, and you may be able to find alternatives to the various MATLAB toolboxes. Here's something that may be helpful:
Luigi Giaccari has a few highly-rated submissions related to KNN searching here, here, and here.
Another alternative for a simpler function like MAPSTD is to try and implement a stripped-down version of it yourself. Here's some sample code that replicates the basic behavior of MAPSTD:
M = magic(5); %# Create an example matrix
rowMeans = mean(M,2); %# Compute the row means
rowStds = std(M,0,2); %# Compute the row standard deviations
rowStds(rowStds == 0) = 1; %# Set standard deviations of 0 to 1
for i = 1:size(M,1)
M(i,:) = (M(i,:)-rowMeans(i))./rowStds(i); %# Normalize each row of M
end
%# Or you could avoid the for loop with a vectorized solution...
M = bsxfun(#rdivide,bsxfun(#minus,M,rowMeans),rowStds);
This obviously won't cover all of the options in MAPSTD, but captures the basic functionality. I can confirm that the above code gives the same result as mapstd(M).
You might want to consider getting your own copies of Matlab and the toolboxes you need. Mathworks has VERY attractive pricing for University students.
GNU Octave is the free Matlab more-or-less-work-alike. I don't know how well the toolboxes are covered.
Alternatively, if the assignment needs them, the school probably has them on some lab machines somewhere, and you MIGHT be able to login remotely, using an Xterm of some kind. Ask the TA.
You could also look at R which is very strong in many data-driven fields, including Machine Learning.
Also of note is the MLOSS directory of open-source machine learning software.
Related
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I'm trying to learn APL (Kona), and I'm looking for example projects so I can get an understanding of how an experienced APL'er would organize his/her code.
Any open source projects would be helpful but, non-financial or anything lacking heavy math would be awesome.
I've been looking too, and haven't found any yet. APL is a very old language; it completely predates the whole open-source movement. Unlike equally-old Lisp (whose history includes much AI lab research and a spirit of open collaboration), APL's culture has been historically associated with IBM, commercial timesharing systems, and finance. Kevin and I are trying to change this with Kona, though.
There is a fair bit of k code at no stinking loops. Some of it was written for a different version of k than what Kona targets, though. Hakan Kjellerstrand also has an excellent K page.
There are also several great APL/J/K/Q books. I particularly recommend Kenneth Iverson's A Programming Language, Henry Rich's J for C Programmers, Jeffry Borror's Q for Mortals (Q is the newest version of Kx's K), and Gilman & Rose's APL: An Interactive Approach. All but the last are readily available online.
Keep in mind is that many people are using APLs as mathematical tools (like R, mathematica, gnuplot, etc.) rather than for programming, per se. (IMHO, J is best for that.) K is designed to be a more general-purpose programming language, and feels like a synthesis of APL, Lisp, and C. It is an outlier in the language family, though.
If you read this, you will see that Kona is "an open-source implementation of the K programming language (K3.2)". Unfortunately for you, if you visit the homepage for the creator of the K Programming Language and look under products, you will see that K seems to be unsupported.
Also, looking at the Wikipedia articles for both APL and K, the syntax seems to be quite complex (e.g. x#>#:'x is used to sort a list of strings by their length)! I recommend learning Assembler (through nasm; one of the most popular assemblers), C++, C, and maybe Python rather than APL or K (both seem to be unsupported and unused).
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
Is there any open source alternative for Zonal Statistics tool (ArcGIS Spatial Analyst)?
What is the best tool (which I can use in script) dor counting statistics of raster files?
You can do this with GRASS using various methods. Which one is most suitable will depend on your data and the required output. Note that you can use GRASS also from within QGIS using the GRASS toolbox or Sextante toolbox.
Let's assume you have:
a vector map, e.g., vector_zones with the zones defined in the
column myzones in the attribute table.
a raster layer 'values' for which you want to calculate your zonal statistics
r.statistics
To use r.statistics, you first need to convert the vector map to a raster layer, which you can do with v.to.rast. Next, use r.statistics to calculate the zonal statistics.
v.to.rast input=vector_zones output=zones column=myzones
r.statistics base=zones cover=values out=outputmap method=average
This will give you a new layer with the selected zonal statistic, which could be average, mode, median, variance, etc. (see the man page link above).
r.univar
The r.univar function also works on raster layers.
v.to.rast input=vector_zones output=zones column=myzones
r.univar map=values zones=zones output=output.file fs=;
The output is a table with the zonal statistics.
v.rast.stats
This does not require you to convert the vector layer to a raster layer (this is done internally). The function calculates basic univariate statistics per vector category (cat) from the raster map.
v.rast.stats vector=vector_zones layer=1 raster=values column_prefix=val
The results are uploaded to the vector map attribute table.
you can use the raster package in R
library(raster)
v <- raster('raster filename')
z <- raster('zones raster filename')
zv <- zonal(v, z, fun=mean)
Correct me if I'm wrong, RobertH, but I believe zonal() requires that the zones are already 'rasterized' in some sense, whereas many times one will want the statistics of raster cells that fall within polygons. The various overlay methods in R within the sp package (see: ?"overlay-methods" ) are necessary for this, though if I am wrong I would be glad to hear it. I quite prefer the raster package over using SpatialGridsDataFrames, but I think one must rely on sp classes to mix polygons and gridded data. Which is ok, except becomes problematic because it lacks the great memory management of the raster package, which making point-in-polygons style operations really hard to do in R on large rasters.
I am also led to believe, but have not tried, that this can be done within GRASS, and/or through QGIS, with the next release of QGIS (1.7) to have some sort of built-in zonal stats feature.
The Rasterstats package is a nice open source tool that worked well for me:
http://blog.perrygeo.net/2013/09/24/python-raster-stats/
I started using it as a work-around because arcpy's ZonalStatistics method was producing a problematic raster that lead to an odd error when trying to convert the raster to an array (https://gis.stackexchange.com/questions/110274/save-fails-on-raster-object-created-from-numpyarraytoraster). Rasterstats worked well and provided an efficient solution for my problem.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
Reading well-written code seems to help me learn a language. (At least it worked with C.) [deleting the 'over-specified' part of the question]
I'm interested in particular in lisp's reputation as a language suited to creating a mini-language or DSL specific to a problem. The program ought to be open-source, of course, and available over the web, preferably.
I've Googled and found this example:
http://lispm.dyndns.org/news?ID=NEWS-2005-07-08-1
Anybody have another? (And, yes, I will continue reading "Practical Common Lisp".)
After 11 hours (only 11 hours!): Thanks, everyone. What a wonderful site, and what a bunch of good answers and tips!
I feel your constraints are over-specified:
small enough to comprehend, varied
enough to show off most of (c)lisp's
tricks and features without being
opaque (the 'well-written' part of the
wish), and independent of other
packages.
Common Lisp is a huge language, and the power set that emerges when you combine the language elements is much larger. You can't have a small program showing "most tricks" in CL.
There are also many concepts that you will find alien when you learn CL coming from another language. As such CL is less about tricks but more about its fundamental paradigms.
My suggestion is to read up on it a bit first and then start building your own programs or looking into open source code.
Edi Weitz for example usually writes good code. Check out his projects at http://www.weitz.de/.
And now go read PCL. :)
I'm kind of lazy to find the links, but you should be able to 'Google'/'Bing' it. The following list mentions very different ways to embed languages and very different embedded languages.
ITERATE for iterations
System/Module/File description in 'defsystem's, an example would be ASDF
infix readmacro
define-application-frame in CLIM for specifying user interfaces
embedded Lispified SQL queries in LispWorks and CLSQL
Knowledgeworks of LispWorks: logic language with rules, queries, ...
embedded Prolog in Allegro CL
embedded HTML in various forms
XMLisp, integrates XML and Lisp
Screamer for non-deterministic programming
PWGL, visual programming for composing music
Note that there are simple embedded languages and really complex ones that are providing whole new paradigms like Prolog, Screamer, CORBA, ...
If you haven't taken a look at it yet, the book Practical Common Lisp is available free online and has several example projects.
The LOOP macro is an almost perfect example of a DSL embedded in Common Lisp. However, since it's already part of the standard, it may not be what you're after.
CLs format function have a mini dsl.
http://cybertiggyr.com/fmt/
I think that dsl for printing strings will compile to machine code.
(format nil "~{~A~#[~:;, ~]~}" lst))
CLSQL provides a Lispy notation for SQL queries, which it compiles to SQL, and just about all Lisp HTML and XML generation libraries qualify. Metabang bind is a DSL for lexically binding variables. You probably didn't know you needed one, but it turns out to be amazingly useful.
SERIES is kind of a DSL, depending on your definition. It's in an appendix to CLTL2, though it's not actually part of the language.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
What's a good library for geospatial functions? I mean things like
distance between two points on the globe
coordinates of a circle of a given radius from a particular point
etc.
Bonus if there's an interface to the various ways different databases
represent geolocations.
I'm a geo-noob (in case this question didn't make it obvious), so pointers to other geolocation/geospatial resources are welcome.
C++ and Python preferred, but all pointers welcome.
I've enjoyed using geopy. It's a simple library that finds great-circle distance in a number of projections. Geopy also provides a single interface to multiple geocoders like Google Maps and Microsoft Earth to give you coordinates for a street address.
You might be interested in the Topic :: Scientific/Engineering :: GIS section in PyPi.
Some options for functions from a useful article on the O'Reilly website. There are other options in the article.
GEOS open source C++ geometry /
topology engine. Might suit you?
PostGIS - a PostgreSQL database
that can also store geometric
(spatial) data types. This provides
GIS-like abilities within an SQL
database environment, so you could do manipulations through SQL.
I'm not sure about interfaces to different databases but the article mentions a number of libraries that convert geospatial data between different formats. You might also be interested in the OGC standards. If all your target databases support WFS you should be able to access them with exactly the same code. EDIT: Increasing numbers of GIS packages support WFS but I don't think pure databases do.
EDIT: you could also check out OSGeo which is a collaboration to support open source geospatial stuff. FDO might do the interfaces to different databases.
If you were using ruby with or without rails, I'd recommend the GeoKit gem: http://geokit.rubyforge.org/
Check GDAL/OGR which is two libraries in one - GDAL for raster data and OGR for vector data. The OGR part provides API to manipulate geometries - it uses GEOS library.
I was looking at something similar, there's the Geolocator Library that I found, it provides some distance calculation functions.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I'm trying to find some good examples of semantic diff/merge utilities. The traditional paradigm of comparing source code files works by comparing lines and characters.. but are there any utilities out there (for any language) that actually consider the structure of code when comparing files?
For example, existing diff programs will report "difference found at character 2 of line 125. File x contains v-o-i-d, where file y contains b-o-o-l". A specialized tool should be able to report "Return type of method doSomething() changed from void to bool".
I would argue that this type of semantic information is actually what the user is looking for when comparing code, and should be the goal of next-generation progamming tools. Are there any examples of this in available tools?
We've developed a tool that is able to precisely deal with this scenario. Check http://www.semanticmerge.com
It merges (and diffs) based on code structure and not using text-based algorithms, which basically allows you to deal with cases like the following, involving strong refactor. It is also able to render both the differences and the merge conflicts as you can see below:
And instead of getting confused with the text blocks being moved, since it parses first, it is able to display the conflicts on a per method basis (per element in fact). A case like the previous won't even have manual conflicts to solve.
It is a language-aware merge tool and it has been great to be finally able to answer this SO question :-)
Eclipse has had this feature for a long time. It's called "Structure Compare", and it's very nice. Here is a sample screenshot for Java, followed by another for an XML file:
(Note the minus and plus icons on methods in the upper pane.)
To do "semantic comparisons" well, you need to compare the syntax trees of
the languages, and take into account the meaning of symbols. A really
good semantic diff would understand the language semantics, and realize
when one block of code was equivalent in function to another. Going
this far requires a theorem prover, and while it would be extremely
cute, isn't presently practical for a real tool.
A workable approximation of this is simply comparing syntax trees, and reporting
changes in terms of structures inserted, deleted, moved, or changed.
Getting somewhat closer to a "semantic comparison", one could report
when an identifier is changed consistently across a block of code.
See our http://www.semanticdesigns.com/Products/SmartDifferencer/index.html
for a syntax tree-based comparison engine that works with many languages, that does
the above approximation.
EDIT Jan 2010: Versions available for C++, C#, Java, PHP, and COBOL.
The website shows specific examples for most of these.
EDIT May 2010: Python and JavaScript added.
EDIT Oct 2010: EGL added.
EDIT Nov 2010: VB6, VBScript, VB.net added
What you're groping for is a "tree diff". It turns out that this is much harder to do well than a simple line-oriented textual diff, which is really just the comparison of two flat sequences.
"A Fine-Grained XML Structural Comparison Approach" concludes, in part with:
Our theoretical study as well as our experimental evaluation
showed that the proposed method yields improved structural similarity results with
respect to existing alternatives, while having the same time complexity (O(N^2))
(emphasis mine)
Indeed, if you're looking for more examples of tree differencing I suggest focusing on XML since that's been driving practical developments in that area.
Shameless plug for my own project:
HTML Tree Diff does structure-aware comparison of xml and html documents, written in python.
http://pypi.python.org/pypi/html-tree-diff/0.1.0
The solution to this would be on a per language basis. I.e. unless it's designed with a plugin architecture that defers a lot of the parsing of the code into a tree and the semantic comparison to a language specific plugin then it will be very difficult to support multiple languages. What language(s) are you interested in having such a tool for. Personally I'd love one for C#.
For C# there is an assembly diff add-in to Reflector but it only does a diff on the IL not the C#.
You can download the diff add-in here [zip] or go to the project on the codeplex site here.
A company called Zynamics offers a binary-level semantic diff tool. It uses a meta-assembly language called REIL to perform graph-theoretic analysis of 2 versions of a binary, and produces a color-coded graph to illustrate differences between them. I am not sure of the price, but I doubt it is free.
http://prettydiff.com/
Pretty Diff minifies each input to remove comments and unnecessary white space and then beautifies the code prior to the diff algorithm. I cannot think of anyway to become more code semantic than this. And, its written JavaScript so it runs directly in the browser.