Summary: It is often remarked that first-order Peano Arithmetic is non-categorical but deductively well-behaved, while second-order Peano Arithmetic is categorical but deductively ill-behaved. This suggests that, when it comes to axiomatizations of mathematical theories, expressive power and deductive power may be orthogonal, mutually exclusive desiderata. In this paper, I turn to J. Hintikka’s distinction between descriptive and deductive approaches in the foundations of mathematics [“Is there completeness in mathematics after Gödel?”, Philos. Top. 17, No. 2, 69–90 (1989; doi:10.5840/philtopics19891724)] to discuss the implications of this observation for the first-order logic versus second-order logic divide. The descriptive approach is illustrated by Dedekind’s ‘discovery’ of the need for second-order concepts to ensure categoricity in his axiomatization of arithmetic; the deductive approach is illustrated by Frege’s Begriffsschrift project. I argue that, rather than suggesting that any use of logic in the foundations of mathematics is doomed to failure given the impossibility of combining the descriptive approach with the deductive approach, what this apparent predicament in fact indicates is that the first-order versus second-order divide may be too crude to investigate what an adequate axiomatization of arithmetic should look like. I also conclude that, insofar as there are different, equally legitimate projects one may engage in when working on the foundations of mathematics, there is no such thing as the One True Logic for this purpose; different logical systems may be adequate for different projects.