Text, yes. Graphics? SVGs are not as small as people think especially if they're any more complex than basic shapes, and there are plenty of things that simply cannot be represented as vector graphics anyway.
It's fair to prefer text-only pages, but the "and graphics" is quite unrealistic in my opinion.
How much is gained by using SVG (as opposed to a raster graphics format) varies a lot depending on the content. For some files (even with complex shape paths depending on a couple details) it can be an enormous gain, and for some files it can indeed be disappointing.
That being said, while raw SVG suffers in that respect from the verbosity of the format (being XML-based and designed so as to be humanly readable and editable as text), it would be unfair to compare, for the purpose of HTTP transmission, the size of the raster format image (heavily compressed) with the size of the SVG file (uncompressed) as one would if it were for desktop use. SVG tends to lend itself very well to compressed transmission, even with high-performance compression algorithms like brotli (which is supported by all relevant browsers and lots of HTTP servers), and you can use pre-compressed files (e.g. for nginx with the module ngx_brotli) so that the server doesn't have to handle compression ad hoc.
It's fair to prefer text-only pages, but the "and graphics" is quite unrealistic in my opinion.