Screw whatever other people might think. It simply doesn’t matter. You’re happy, your wife is happy, your child will be happy. That’s all that matters.
For us, it is the exact opposite - we decided we didn’t want to have children. People keep asking why not, and try to convince us that children are sort of a must have in our age. But you know what? Screw them - our lifes, our decision.
Is it your dream house? Then buy. Is it not? Then don’t.
Opportunities come and go. You have to decide which to pursue.