Well, not every time.
Sometimes you should use an apostrophe to make a plural. Here’s a note from the Chicago Manual of Style and Daily Writing Tips:
DO use the apostrophe to form the plural of an abbreviation that combines upper and lowercase letters or has interior periods:
The department graduated five M.A.’s and two Ph.D.’s this year.
NOTE: If you leave out the periods, you can write MAs but you’d still have to write PhD’s.
DO use the apostrophe to form the plural of lowercase letters:
Mind your p’s and q’s.
DO NOT use the apostrophe to form the plural of capital letters
Apostrophes being used incorrectly is a growing phenomenon that drives us up the wall. This is a good guide. :)