Seems reasonable to me that since the brain is very complex and genes that affect structure and development would give some individuals advantages or disadvantages. I mean if early childhood nutrition can influence intellectual ability, genetic factors playing a role isnt that ridiculous.

You made the claim that IQ or general intellectual ability is not heritable, but this was widely regarded as true not that long ago. Please cite sources for your claim.