Like all mammals, guinea pigs drink their mother's milk for the first few days, gradually introducing and then moving over to solids. If you have a very young baby guinea pig who has been rejected by its mother, you will need to hand-feed it.
Hand-feeding should be done with a teaspoon rather than with a syringe. Baby guinea pigs can easily inhale the liquid in the syringe by accident and choke. Use full-fat goat’s milk, or prepare a milk mixture of half water and half evaporated milk. Soak brown bread in the milk, and offer this to the baby GP on a teaspoon - they’ll suck the milk out.
This isn't as nutritious as a guinea pig mother's milk, but it's the second best thing. The aim is to get the baby through to the solids stage.
Baby guinea pigs are sometimes rejected by their moms
If a baby guinea pig has been rejected by its mother, you will need to feed it every one to one-and-a-half hours. Let them take as much of the milk as they want at each feeding time.
If the baby has been completely rejected (rather than just having a mom who refuses to let it suckle), it can quickly become cold, away from its mother's fur. In these circumstances you'll need to keep the young GP warm with a hot water bottle wrapped in blankets (so that it’s not too warm).
You will need to stimulate the baby GP to go to the bathroom during the first three or four days, as they can’t do this without help. Mom would normally give them an encouraging lick, and this makes them open their bladder and bowels. So, once you’ve fed them, gently wipe their genital region with a cotton bud to encourage them.