Bacterial cells navigate their environment by directing their movement along chemical gradients. This process, known as chemotaxis, can promote the rapid expansion of bacterial populations into previously unoccupied territories. However, despite numerous experimental and theoretical studies on this classical topic, chemotaxis-driven population expansion is not understood in quantitative terms. Building on recent experimental progress, we here present a detailed analytical study that provides a quantitative understanding of how chemotaxis and cell growth lead to rapid and stable expansion of bacterial populations. We provide analytical relations that accurately describe the dependence of the expansion speed and density profile of the expanding population on important molecular, cellular, and environmental parameters. In particular, expansion speeds can be boosted by orders of magnitude when the environmental availability of chemicals relative to the cellular limits of chemical sensing is high. Analytical understanding of such complex spatiotemporal dynamic processes is rare. Our analytical results and the methods employed to attain them provide a mathematical framework for investigations of the roles of taxis in diverse ecological contexts across broad parameter regimes.