The Cramér-Rao bound serves as a crucial lower limit for the mean square error of an estimator in frequentist parameter estimation. Paradoxically, it requires highly accurate prior knowledge of the estimated parameter for constructing the optimal unbiased estimator. In contrast, Bhattacharyya bounds offer a more robust estimation framework with respect to prior accuracy by introducing additional constraints on the estimator. In this work, we examine divergences that arise in the computation of these bounds and establish the conditions under which they remain valid. Notably, we show that when the number of constraints exceeds the number of measurement outcomes, an estimator with finite variance typically does not exist. Furthermore, we systematically investigate the properties of these bounds using paradigmatic examples, comparing them to the Cramér-Rao and Bayesian approaches.