My kids grew up in an Early learning Center so have managed to miss almost no school. My dd missed 1/2 day of Kindergarten, 0 days in grade one and so far 0 in grade 2.
My son just recently missed 1/2 day of Kindergarten.
I think if your child is sick than he is sick and you are looking out for his physical well being. I don't think it will negatively impact him. I think if you were just letting him miss school for no reason it would be different