There used to be a store in north county that sold our heavy t-shirts but I believe many specialty stores have gone out of business.
You could try Clairemont Surf & Sport, Encinitas Surf or Hansens. I know a few in south Orange County - namely one at the Dana Point Harbor called Top Brass.
Hope this helps