MR Tractography, which is based on MRI measures of water diffusivity, is currently the only method available for noninvasive reconstruction of fiber pathways in the brain. However, it has several fundamental limitations that call into question its accuracy in many applications. Therefore, there has been intense interest in defining and mitigating the intrinsic limitations of the method. Recent studies have reported that tractography is inherently limited in its ability to accurately reconstruct the connections of the brain, when based on voxel-averaged estimates of local fiber orientation alone. Several validation studies have confirmed that tractography techniques are plagued by both false-positive and false-negative connections. However, these validation studies which quantify sensitivity and specificity, particularly in animal models, have not utilized prior anatomical knowledge, as is done in the human literature, for virtual dissection of white matter pathways, instead assessing tractography implemented in a relatively unconstrained manner. Thus, they represent a worse-case scenario for bundle-segmentation techniques and may not be indicative of the anatomical accuracy in the process of bundle segmentation, where streamline filtering using inclusion and exclusion regions-of-interest is common. With this in mind, the aim of the current study is to investigate and quantify the upper bounds of accuracy using current tractography methods. Making use of the same dataset utilized in two seminal validation papers, we show that prior anatomical knowledge in the form of manually placed or template-driven constraints can significantly improve the anatomical accuracy of estimated brain connections. Thus, we show that it is possible to achieve a high sensitivity and high specificity simultaneously, and conclude that current tractography algorithms, in combination with anatomically driven constraints, can result in reconstructions which very accurately reflect the ground truth white matter connections.