The composition of the Gram-positive cell wall is typically described as containing peptidoglycan, proteins and essential secondary cell wall structures called teichoic acids, which comprise approximately half of the cell wall mass. The cell walls of many species within the genera Streptococcus, Enterococcus and Lactococcus contain large amounts of the sugar rhamnose, which is incorporated in cell wall-anchored polysaccharides (CWP) that possibly function as homologues of well-studied wall teichoic acids (WTA). The presence and chemical structure of many rhamnose-containing cell wall polysaccharides (RhaCWP) has sometimes been known for decades. In contrast to WTA, insight into the biosynthesis and functional role of RhaCWP has been lacking. Recent studies in human streptococcal and enterococcal pathogens have highlighted critical roles for these complex polysaccharides in bacterial cell wall architecture and pathogenesis. In this review, we provide an overview of the RhaCWP with regards to their biosynthesis, genetics and biological function in species most relevant to human health. We also briefly discuss how increased knowledge in this field can provide interesting leads for new therapeutic compounds and improve biotechnological applications.