Sample size doesn't scale linearly with population size, it converges as population approaches infinity. The harder issue is getting an unbiased sample.