Sam, Gilly, and baby Sam all arrived in Oldtown on the season six finale. The Citadel is located in Oldtown, which is where Sam is trying to be granted access for maester-training. The only hitch is that women and children aren't allowed in the Citadel, so Gilly and baby Sam will need to find somewhere else to live.

Don't forget that Sam and company stopped at Horn Hill, House Tarly's castle, on their way. There they met Sam's cruel father, who threatened to kill Sam for bringing a wildling to his home. Sam and Gilly left in the middle of the night, and Sam stole his father's Valyrian steel sword Heartsbane on the way out.