Baby Yoshis first appeared in Super Mario World, a launch title for the SNES, which was also the game that debuted Yoshi. The Baby Yoshies are found in Red, Blue, or Yellow eggs. You can make them grow into adults by feeding it 5 enemies. You could also feed it a Mushroom, Fire Flower, Feather, or Star to have it grow instantly. Also, Bowser has trapped quite a few of these babies as well, and Mario plans to save them.
Green Baby Yoshis also seem to appear but only in the "Thank You" screen when beating Super Mario World for the SNES and in the end of Yoshi's Island DS. In the later game, it was revealed that a green Baby Yoshi was one of the seven star children. It has been speculated that this Yoshi is in fact the one that appears in most of the Mario video games that don't take place in the past.