Today, I read Toda’s 1989 paper. Titled “PP is as hard as the polynomial-time hierarchy”, Toda proved that \(PH \subseteq P^{\#SAT}\). This is quite the strange theorem since we know that \(P^{SAT} = {\Delta}_{2}^{P}\), which is the set of languages contained within \(PH\) at the second level. \(\#SAT\) and \(SAT\) are closely related, so why does one give so much more computational power than the othe (or in a different perspective – why is alternation weaker than we had believed!)? Maybe one insight is that instances of \(\#SAT\) could have exponentially many satisfying models (to which the \(\#SAT\) oracle can know right away).

However, this is still strange to me. For one, \(M_{SAT}\) still needs to comb through an exponential number of models in the worst case to find a solution (unless \(P=NP\)), so it still encodes some exponential amount of computation. Hence, the fact that \(P^{SAT}\) is contained in \(PH\) but \(P^{\#SAT}\) is not suggests that there’s something weirder going on here. In my opinion, the key intuition here is that there is a “double” barrier against heuristics that machines computing \(\#SAT\) cannot exploit. The decider for \(SAT\) could get lucky and find a model for a formula quickly, but the same probably doesn’t apply for \(\#SAT\) unless \(PH\) collapses. Deciders for \(\#SAT\) can get lucky in finding one model, but would still need to count the (potentially exponential in the worst case) number of satisfying models to the formula. This might also suggest that \(NP\) can be characterized as the set of languages which have a high variance of problem difficulty¹ (and thus heuristics can work well) whereas \(\#P\) can be characterized as the set of languages which have low variance of problem difficulty, if at all (where heuristics tend to fail).

Hence, a program which takes advantage of being able to encode some exponential amount of computation might just be powerful enough to compute languages more powerful than those at arbitrary levels of the polynomial hierarcy (which can only support a constant number of alternations). This also creates an interesting point of view for computational resources: if something encodes “exponential” of some resource, then there’s probably a hacky way to use its oracle to do some crazy things, like making P with a \(\#SAT\) oracle as hard as \(PH\)!

References

Toda’s 1989 Paper