“These small recesses found all over our house.”
Answer: "There used to be open flame burners that connected to gas as a heat source in the early 1900s before forced air, and sometimes they would be set into the wall like this."
I have one that has been in use for over 80 years, there is nothing missing on it.
Driveway marker