For R is simply about creating algorithmic representations of numerical formulas.
R language is beautifully pure. You do not have to worry about all the overhead present in general-purpose languages like C++ or Java. There is no user interface to design either.
Probably the most serious R competitor is Matlab.
Since Matlab is a proprietary software, the development of cutting-edge tools seems to lag behind R.
However, there is an open source Matlab alternative available (for Linux and OS X, but unfortunately not for Windows): GNU Octave.
And the number of packages for GNU Octave has been (rather slowly...) raising: http://octave.sourceforge.net/packages.php
(although it seems development of many packages stopped in 2009)
I wasn't able to find an actively maintained IDE for GNU Octave (similar to R studio for R).
At least you can easily get Octave syntax highlighting in gedit.
The biggest shortcoming of both R and GNU Octave is probably their debugging capability. In case of R you can try Revolution R Enterprise from Revolution Analytics.
To familiarize yourself with GNU Octave you can start with Introduction to Octave. Much more details can be found in Octave online documentation.
R users should probably consult a list of similarities and differences between R and Octave, available in R for Octave users.
There is also a number of blogs about GNU Octave, conveniently aggregated at http://planet.octave.org/
And when you need a physical guide, you can read "GNU Octave Beginner's Guide" by Jesper Schmidt Hansen.
The most recent newcomer to the open source scientific computing area is Julia.
According to the information on the Julia's site:
"Julia is a high-level, high-performance dynamic programming language for technical computing (...)
It provides a sophisticated compiler, distributed parallel execution... Julia itself, also integrates mature, best-of-breed C and Fortran libraries for linear algebra, random number generation, FFTs, and string processing. (...)
"The syntax of Julia is similar to MATLAB®"
Julia is presumably faster than both R and Matlab and GNU Octave.
But its future is uncertain at this moment...
You can find some links to Julia resources at: http://www.statalgo.com/julia/