According to Joe Pesci, improvisation and ad-libbing came out of rehearsals, where Martin Scorsese let the cast do whatever they wanted. He made transcripts of these sessions, took the lines that he liked best, and put them into a revised script, which the cast worked from during principal photography.