Summary: Chance constraints are an important modeling tool in stochastic optimization, providing probabilistic guarantees that a solution “succeeds” in satisfying a given constraint. Although they control the probability of “success,” they provide no control whatsoever in the event of a “failure.” That is, they do not distinguish between a slight overshoot or undershoot of the bounds and more catastrophic violation. In short, they do not capture the magnitude of violation of the bounds. This paper addresses precisely this topic, focusing on linear constraints and ellipsoidal (Gaussian-like) uncertainties. We show that the problem of requiring different probabilistic guarantees at each level of constraint violation can be reformulated as a semi-infinite optimization problem. We provide conditions that guarantee polynomial-time solvability of the resulting semi-infinite formulation. We show further that this resulting problem is what has been called a comprehensive robust optimization problem in the literature. As a byproduct, we provide tight probabilistic bounds for comprehensive robust optimization. Thus, analogously to the connection between chance constraints and robust optimization, we provide a broader connection between probabilistic envelope constraints and comprehensive robust optimization.