The relationship between the higher-order factor model and the hierarchical factor model is explored formally. We show that the Schmid-Leiman transformation produces constrained hierarchical factor solutions. Using a generalized Schmid-Leiman transformation and its inverse, we show that for any unconstrained hierarchical factor model there is an equivalent higher-order factor model with direct effects (loadings) on the manifest variables from the higher-order factors. Therefore, the class of higher-order factor models (without direct effects of higher-order factors) is nested within the class of unconstrained hierarchical factor models. In light of these formal results, we discuss some implications for testing the higher-order factor model and the issue of general factor. An interesting aspect concerning the efficient fitting of the higher-order factor model with direct effects is noted.