Building "binders of lore", while usually important, is actually much easier than actually writing the final product, and it's really not analogous to a spec (that would be your detailed outline). Background material is fun and feels productive, but can easily lead you around your goal rather than toward it. Maybe you operate differently, but as someone with a tendency to build worlds rather than plots, that part made perfect sense to me.