Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Lately I was entertaining an idea of how this whole thing may be improved, but I don't really see how can I possibly influence it, so let me splash it out here.

What do we have now:

1. We have a common platform, which is a de-facto standard for sharing (somewhat math-related) papers: arxiv.org. Basically, anyone can upload anything to arxiv.org, there's not much of quality review required. Which, all in all, I think is great. I mean, it's orders of magnitude better than other fields, where both publishing and accessing a published paper is expensive, painstakingly slow and inconvenient process which is obviously great for the journals that have a monopoly over it, but less so for everyone else. All being justified by the need of peer-review (which is not even sure to be working alright, because (1) we all know the quality of the vast number of studies in some social field is shit, and (2) you can even publish a completely bullshit paper in a very reputable journal if you try hard enough).

2. Much of the research is done in the academia (because, well, that's supposedly the reason it exists), but, paradoxically, the main moving force of the progress isn't really interested in progress. It is a competitive environment, so everybody is interested in publishing a paper which will be recognized as an "important one" later on, so the. They are not interested in anyone learning anything and especially not interested in making their rivals (and that is what any other researcher is, a rival) able to make a follow-up contribution easily.

3. Hence, many important studies by reputable scientists lack quality, which everyone knows, including the scientists. Papers are written in a passive-agressive tone, in particularly confusing terms an inconvenient format, they often lack the source code and the source data, a minor one-sentence contribution to the current state-of-the-art is spread around 20 pages which you have to read fully to understand what exactly the contribution is.

4. To summarize: everyone is interested in publicity of their work, nobody is interested in improving their work because it provides them the high ground, but everyone (including even those outside of the academia) is interested in improving others work, since it helps them to learn. Improvements on the others work, obviously, are not shared unless it can be framed as a follow-up study.

How (I think) it might be improved. Assume a platform everyone uses to share their work, let's call it arxiv2. arxiv2 has a ranking system, which pushes the papers it "thinks" are of the better quality to the top. "Quality" is (maybe not exhaustively, but that's the main idea) a set of domain-specific flags which are considered to be beneficial for a study to be conclusive in this field, like "has a source code", "has a source data", "has a benchmark/accuracy/specificity/precision metrics", "summary/abstract succinctly captures the main result". Depending on the field, features may be more complicated in order to address some common problems like p-hacking or non-representative sampling in the study. But all the metrics have to be concrete. Nobody is required to include anything, but if they don't, and if others mark the work as lacking some features, it has less chance to get to the top. Everyone can vote for the paper, some minimal moderation may take place. Disputes may be solved similarly to how they are solved on wikipedia/stackoverflow (which are, admittedly, awful and frustrating, but order of magnitudes better than the status-quo in the research).



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: