It's supposed to be the sharp symbol; it's just that it was a hassle for them to use it consistently in paths etc, so they defaulted to # as a stand-in.

It's "sharp" (i.e. higher tone) because it's a higher-level language compared to C and C++.