~30 years old here, and I can confirm I risked and suffered data loss multiple times. I remember when I still was at my parents house during university I had in mind to build a NAS to properly host the family's photo archive (main reason was bit rot protection, which we experienced, but I was planning for a proper backup as well). But I kept procrastinating. Then, I risked losing it due to a distraction. After spending one week to recover it, building a proper solution to host and backup it became top of my list.
Unfortunately, your comment reminds me that car manufactures aren't Google or Apple either, and still it seems that modern, high-end cars have a tendency of collecting more than what's needed for the car to remain in good shape or be diagnosed. I guess we became so defeatist when more and more companies started realizing they could make extra profit by collecting data from their customers.
If you have lots of data, I would suggest using a filesystem designed for this, like ZFS or BTRFS. You would still have to spend in storage though, as ultimately protecting from bit rot requires redundancy.
I do use btrfs, but I don't really understand how to use it to protect against bit rot. I tried to look into it at some point, but got overwhelmed. Did invest in two backup copies, one of them off site, though.
However, looking at prices, I can see one of those backups just being dismantled to be used as more storage.
If you got overwhelmed in trying to understand how to do it, just do it the easy way: create a ZFS pool of two drives of identical size in mirroring. Then, make sure to have regular scrubs (this means ZFS will regularly check all files for corruption, and recover from eventual errors), use your OS' task scheduler (e.g. Systemd or cron) to run this once a month. This should be sufficient to protect you from bit rot.
I've currently got two 10TB, one 14TB and one 18TB drive. I don't think what you describe is as easy for me as you expect. I've got one backup in a different city and one in cold storage.
The sizes are all over the place because I only buy a new drive when I run out of storage and then just get the largest one within a reasonable cost. It has been cost prohibitive to mirror everything and current prices aren't encouraging. Neither is the rate at which HDDs get larger. I was kinda hoping for 100TB drives under $400 by 2025 back in 2014, but that hope has been shattered.
Zfs and btrfs will avoid corruption caused below the file system layer, not something like malware or misuse from a tired sysadmin. I run a tool called integrit that creates checksums and lets me detect non-wanted changes. I only run it on important stuff that does not change too much like photos and videos. Not fun to have to regularly check the log but it works fairly well. This could maybe be extended with automatic checking of the logs.
I also struggle to reconcile these things, I guess the only way would be to try and see if it works for me, without caring too much if it works for other people on the internet. I would just like to add that, DwarfStar's author (Salvatore Sanfilippo) is a strong supporter of the idea that you shouldn't read the code, and he says he never read DwarfStar's code. And still, it seems that this project is much more than PoC and actually both usable and useful for people doing local inference (I didn't try it myself, but I saw a lot of positive comments about it). Could be that the crucial point is in how we use those models: instead of giving it a general goal (e.g. build me an inference engine) Sanfilippo, being an experienced programmer, kept pointing the models in the right direction. He also read the papers related to the models he was programming support for in DwarfStar, so that, when he worked on optimizations, he knew what should be done instead of prompting a general "please optimize this". So, I would say that, if you let the agents work on a "feature by feature" basis instead of trying to on3-shot things, you get much better results. Could also be that, by attempting to one-shot large projects, the model starts coding badly due to context window exhaustion.
Sorry for the not so well written comment, I was just throwing in some ideas.
Yes, you can get better pricing if you do it yourself. But the true advantage of OpenRouter is that, in a space where there's a new model being released every week, you can easily switch to whatever model is best at any given time without having to set up accounts with multiple providers. Or you can just experiment with the latest release. Their product is the convenience. Of course if you decide to only use a specific provider or two, then you don't need OpenRouter.
But it's not really that difficult to make a clone of OpenRouter's service. What they do isn't really that original.
Their only value comes from the fact that the currently have lots of traffic. And I dkn't think that their cumstomers are really bound to theur servuce. They could switch to a competitor without too much hassle.
Yes, but as l9ng as there's no clone, they're good. On the potential clone side, I guess many will be put off by the fact that "there's already OpenRouter". Also, a clone would need to find a way to make existing OpenRouter customers to switch, which isn't easy.
One of my sci-fi future ideas, every time I hear about someone thinking of a new way to make warp drives, is that if we ever sort it out someone will make TARDISes for the filthy rich - a static warp bubble that moves space into a ground floor closet or room from space in an upstairs or in the attic.
Like a milder version of the gimmick in Ultraviolet.
IMO there's a big difference between using AI to write code and using it to review your code. In the latter, you know the code well (you are the one who wrote it), and so it is much easier to understand if AI suggestions are good or hallucinations. In the former case, realizing when the AI is making mistakes is harder, because you don't have the full context anymore.
reply