J. Crew's name set it up to compete with Ralph Lauren's Polo line.
According to Forbes, "The name Crew was picked to compete with Ralph Lauren’s Polo label and [founder Arthur] Cinader added the J because he thought it added [cachet]."
Makes sense. Polo and crew are both pretty preppy sports.