The basic tradeoff is between flexible input and flexible output. I do think it is sound advice to push the relational database model (this overlaps but is not identical to the relational algebra model) to the breaking point before going elsewhere. If you don't do data validation you can't do ad hoc data transformation later reliably. Strict schemas are always an investment in the future for that reason.
> Polymorphic structures are a PITA to model in relational tables, but they fit very well in schemaless, document-oriented NoSQL systems. This has nothing to do with data validation.
JSON and XML types in PostgreSQL rock. If you can limit the areas where the data is polymorphic, you can push the rest into those types. Many other RDBMS's support XML types as well.
> Rapidly changing schemas are a PITA in RDBMSes, especially with a large cluster of servers. It's usually very easy to alter schemas on-the-fly with NoSQL systems. This has nothing to do with data validation.
This again trades one set of problems for another. You are allowing old schema and new schema to co-exist. How many old schemas does your application have to support? It seems to me this would likely ossify things down the road a bit because if your schema is that rapidly changing, you may never get caught up and so backward compatibility is going to be a bitch. In other words, the less guarantees you can make regarding information stored, the fewer guarantees you can make about data output. That's a big deal as your product matures.
JSON and XML types give you all the pain of a NoSQL system with none of the benefits of an RDBMS. No thanks. My tools elegantly map database fields to my objects; doing this with serialization formats is incredibly crude and brittle. I do it for some edge cases already; I am looking forward to removing that code.
A schemaless datastore does not mean a schemaless app. Furthermore, the application layer is a far better place to do validation simply because it has more knowledge of the real world. An RDBMS can constraint an age field to be integer; the application can constrain it to be positive and within the probable lifespan of a human.
How many old schemas does your application have to support? Only as many as I want to; reprocessing removes the old, which I tend to do fairly quickly. I have done a lot of schema migration over the last three years on GAE and the process is vastly easier than it is in RDBMS-land. There's no need to stop the world.
Hey, I love Postgres. I spent most of my (almost two-decade-long) career building apps on many different RDBMSes. But there are many applications that are better suited to alternatives, especially with large-scale consumer-facing web apps. These threads annoy me; we should all be looking at new tools saying "wow, I wonder what this is good for" rather than smugly burning all the heretics that turn away from to the One True Idol.
Typically if you have some polymorphic data attached to some other data, a hybrid approach works. After all, you can't report on what you can't easily quantify.
My tools elegantly map database fields to my objects; doing this with serialization formats is incredibly crude and brittle.
Build tools to do it.
A schemaless datastore does not mean a schemaless app.
That's actually a problem in most cases.
But there are many applications that are better suited to alternatives, especially with large-scale consumer-facing web apps. These threads annoy me; we should all be looking at new tools saying "wow, I wonder what this is good for" rather than smugly burning all the heretics that turn away from to the One True Idol.
Certainly that last part is true, but the best uses I have seen, including uses put forth by NoSQL proponents, have been as adjuncts to a relational approach, for example handling data transformation on the way in or out. Otherwise you end up with two fundamental problems:
1) How many other apps can your application support? (see https://news.ycombinator.com/item?id=4495377) Where is your public API? Basically you can do this in your db layer and/or your app layer, but you get a lot more flexibility by doing it in the db. With a schemaless app, you can't do it there without intimate knowledge of innards...
2) Strict schemas with validated input and a declarative query language buys you flexible output. Rigid schemas are thus an investment in the future. If your data is sufficiently polymorphic that output can't be flexible no matter how you do it, then I suppose that's not a huge issue. However for many cases where NoSQL is chosen wrongly it is a huge issue.
I'm guessing you're a big fan of stored procedures too.
In certain times and places. However, I am not a fan of the idea that all db access goes through stored procedures. LedgerSMB will eventually do everything through stored procs but the reason is to make sure that all the logic is available to other apps, not to require that everyone else do all access through the stored procs.
Also our stored procs are designed to be discoverable by the application and therefore allow for looser coupling between the app and sproc than is typical.
Too late to edit but it is worth noting that stored procedures are also very much all or nothing in terms of quality. Well-writen SP's are a delight to work with. Horribly written ones are infuriating. It's not like other environments where the quality of the code can degrade gracefully.
> Polymorphic structures are a PITA to model in relational tables, but they fit very well in schemaless, document-oriented NoSQL systems. This has nothing to do with data validation.
JSON and XML types in PostgreSQL rock. If you can limit the areas where the data is polymorphic, you can push the rest into those types. Many other RDBMS's support XML types as well.
> Rapidly changing schemas are a PITA in RDBMSes, especially with a large cluster of servers. It's usually very easy to alter schemas on-the-fly with NoSQL systems. This has nothing to do with data validation.
This again trades one set of problems for another. You are allowing old schema and new schema to co-exist. How many old schemas does your application have to support? It seems to me this would likely ossify things down the road a bit because if your schema is that rapidly changing, you may never get caught up and so backward compatibility is going to be a bitch. In other words, the less guarantees you can make regarding information stored, the fewer guarantees you can make about data output. That's a big deal as your product matures.
And don't get me started on BASE.....