Après-ski crewneck sweatshirt

I normally wear a small in all things J. Crew, but I wanted this sweatshirt to be big and baggy, so I ordered a S and a M. Well, the M fits like a normal S. So, no big and baggy Apres Ski sweatshirt for me, but I'm keeping the M and the S is going back. If not for shipping/return charges, I'd prefer a L I'm guessing...