When public debt is elevated, the fiscal cost of fighting inflation rises sharply, as interest rate hikes increase government interest expenditures. We formalize this mechanism in a nonlinear New Keynesian model with a state-dependent fiscal constraint on monetary policy. High debt may dampen the monetary response to inflation, generating an inflationary bias even though government debt remains fully fiscally backed. The interaction between high debt and inflationary cost-push shocks makes the fiscal limit more likely to bind, amplifying inflation. In demand-driven downturns, the fiscal constraint may become more restrictive than the zero lower bound, forcing the central bank to either print money to purchase excess debt or accept fiscal dominance.