Quantifying the microstructural and macrostructural geometrical features of the human brain’s connections is necessary for understanding normal aging and disease. Here, we examine brain white matter diffusion magnetic resonance imaging data from one cross-sectional and two longitudinal data sets totaling in 1218 subjects and 2459 sessions of people aged 50-97 years. Data was drawn from well-established cohorts, including the Baltimore Longitudinal Study of Aging data set, Cambridge Centre for Ageing Neuroscience data set, and the Vanderbilt Memory & Aging Project. Quantifying 4 microstructural features and, for the first time, 11 macrostructure-based features of volume, area, and length across 120 white matter pathways, we apply linear mixed effect modeling to investigate changes in pathway-specific features over time, and document large age associations within white matter. Conventional diffusion tensor microstructure indices are the most age-sensitive measures, with positive age associations for diffusivities and negative age associations with anisotropies, with similar patterns observed across all pathways. Similarly, pathway shape measures also change with age, with negative age associations for most length, surface area, and volume-based features. A particularly novel finding of this study is that while trends were homogeneous throughout the brain for microstructure features, macrostructural features demonstrated heterogeneity across pathways, whereby several projection, thalamic, and commissural tracts exhibited more decline with age compared to association and limbic tracts. The findings from this large-scale study provide a comprehensive overview of the age-related decline in white matter and demonstrate that macrostructural features may be more sensitive to heterogeneous white matter decline. Therefore, leveraging macrostructural features may be useful for studying aging and could facilitate comparisons in a variety of diseases or abnormal conditions.