Abstract: Dirac plasmons in graphene hybridize with phonons of transition metal dichalcogenides (TMDs) when the materials are combined in so-called van der Waals heterostructures (vdWh), thus forming surface plasmon-phonon polaritons (SPPPs). The extend to which these modes are coupled depends on the TMD composition and structure, but also on the plasmons' properties. By performing realistic simulations that account for the contribution of each layer of the vdWh separately, we calculate how the strength of plasmon-phonon coupling depends on the number and composition of TMD layers, on the graphene Fermi energy and the specific phonon mode. From this, we present a semiclassical theory that is capable of capturing all relevant characteristics of the SPPPs. We find that it is possible to realize both strong and ultra-strong coupling regimes by tuning graphene's Fermi energy and changing TMD layer number.