Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Does RethinDB has a concept of transaction? My question is actually about restoring a lost node... If a node is rebooted, will all the data for its shards going to be sent again? Or just the delta?

Similarly if I have to rebuild a node from scratch, is there a way to prime it so that a massive copy of all the data in the cluster gets copied to it from the other nodes?



> If a node is rebooted, will all the data for its shards going to be sent again? Or just the delta?

Just the delta. We built an efficient, distributed BTree diff algorithm. When a node goes offline and comes back up, the cluster only sends a diff that the node missed.

> Similarly if I have to rebuild a node from scratch, is there a way to prime it so that a massive copy of all the data in the cluster gets copied to it from the other nodes?

You don't have to do that, it happens automatically. You can have full visibility and control into what's happening in the cluster -- check out http://rethinkdb.com/docs/system-tables/ for details on how this works.


> You don't have to do that, it happens automatically

Well, in a past life, I used another store that did that automatically, the issue with that is that EITHER it kills the cluster because of read-congestion as it re-builds the "new" node, OR, if you limit the bandwidth for node-building, it takes for ever and a half to rebuild a node which means that you are exposed with one less shard of what was on that node.

What are the chances of a filesystem snapshot to be consistent enough to be used to prime a crashed node? What about restoring backup files from other nodes?


Congestion vs. time is definitely a hard problem. We've done an enormous amount of tuning to make this work, and the upcoming Raft release does even more. This part has been quite solid for a while, so I think you might have a better experience with RethinkDB than what you're used to.

There is currently no other way to prime the node -- I hope we don't have to add it. This sort of functionality should work out of the box.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: