Where can I learn more about the implications, meta discussions, history and the foundations of mathematics?
Is Russell’s Introduction to Mathematical Philosophy a good start?
Philosophy of Mathematics: Selected Readings, edited by Paul Benacerraf and Hilary Putnam, is one of the standard essay collections, and introduces the classical schools: formalism, intuitionism and logicism.
But there are newer points of view. Personally, I liked The Nature of Mathematical Knowledge by Philip Kitcher because of its sophisticated historical point of view. Kitcher’s approach is empiricist.
Another interesting approach is Charles Chihara’s structuralism, as presented in his book Constructibility and Mathematical Existence.
This next one is not a book but it is famous enough that I think it should be mentioned
here. With respect to axiomatic set theory and philosophy, there is the two-part essay
entitled “Believing the Axioms” by Penelope Maddy:
Maddy, Penelope (Jun. 1988). “Believing the Axioms, I”. Journal of Symbolic Logic 53 (2): 481–511.
Maddy, Penelope (Sept. 1988). “Believing the Axioms, II”. Journal of Symbolic Logic 53 (3): 736–764.
If you are interested in the foundations of set theory in particular, there is the classic
book Foundations of Set Theory by Abraham Fraenkel, Y. Bar-Hillel and A. Levy. The
standard system of axiomatic set theory ZF is named after Ernst Zermelo and Abraham Fraenkel.
I think there are probably some good introduction to “classical” philosophy of math that I’m not aware of, but what I find most interesting are modern treatments of philosophy of math.
Lakoff and Nunez’s book called Where Mathematics Comes From: How the Embodied Mind Brings Mathematics into Being uses modern cognitive science to discuss philosophy of math.
David Corfield’s Toward a Philosophy of Real Mathematics is a fascinating look at what mathematicians actually do (as opposed to most philosophy of math done by philosophers that don’t know much math).
I have lots of other suggestions, but I’ll let other people have their say as well (and moreover this wasn’t quite what you were asking for).
Russell’s book is probably not what you’re looking for. Firstly, it is mostly his opinions on the subject, and some of his arguments are surprisingly weak (as I see them anyways). Here is the online version of Russels book. Looking at the content list, it does not seem what you are looking for.
Personally I can recommend “The Mathematical Experience”, by Davis and Hersh. http://www.amazon.com/Mathematical-Experience-Phillip-J-Davis/dp/0395929687 It presents some of the main philosophical views on mathematics. (among other things)
When I took a course on the philosophy of mathematics, our textbooks were Shapiro’s Thinking about mathematics and Parsons’ Mathematical Thought and its Objects. I recommend them both.
Not a book, but a very scholarly overview article on the Philosophy of Mathematics from the Stanford Encyclopedia of Philosophy:
Try also the books by Wilder and Lakatos.
Other’s have suggested very good books for the history and the foundations part of your question, so I will just mention one which I think is a great fun read and addresses some of the “meta discussions” you asked about: Gian-Carlo Rota’s Indiscrete Thoughts. Rota is, to me at least, quite interesting as his book not only gives lots of nice bits of Math history and anecdotes about all of these mathematicians that you might otherwise only think about as names attached to theorems, but also because his position as a philosopher and mathematician makes him talk about ideas related to math that I have never seen discussed elsewhere (the chapters “the pernicious influence of mathematics on philosophy” and “the pernicious influence of philosophy on mathematics” are good examples of this).
Also, of course, Rota is known as quite a character and spends a great deal of this book living up to that–taking joy in arguing against ideas in math and philosophy he doesn’t agree with, and in telling slightly off color stories about Ulam and von Neumann, which does make the book a great deal of fun.
“What is Mathematics, Really?” by Hersh, though the main thrust of his own views is social-constructivism, gives a great summary of the main schools of thought (including the classics of platonism, logicism, formalism and intuitionism). (of course, a lot of philosophy of mathematics is not covered by a ‘school’)
Since no one mentioned it,
The Oxford Handbook of Philosophy of Mathematics and Logic (Oxford Handbooks in Philosophy) by Stewart Shapiro
I’m surprised that this one hasn’t been mentioned yet:
And a classic:
Morris Kline : Mathematical Thought from Ancient to Modern Times, three volumes