Stability is for a large part determined by the entropy of water, which in turn
is determined by the number of waters that can be 'released' when the protein
folds. Residues that are at the surface (and it doesn't matter whether they are
phobic or philic) touch about equally many waters in the unfolded and in the
folded form, so they do not contribute to the change in the entropy of water
upon folding.
Residues in the core, on the other hand, release their waters when the protein
folds. Fobic residues do that just as much as philic ones. But Philic residues
make H-bonds in the unfolded form, and those get lost when the protein folds.