As one of the comments on reddit posts - it's not just big tech companies, but also entire university teams which feel the goalposts moving miles ahead all of a sudden. Imagine working on your PhD on chat bots since start of 2022. Your entire PhD topic might be irrelevant already...
>Imagine working on your PhD on chat bots since start of 2022. Your entire PhD topic might be irrelevant already...
In fairness most PhD topics people work on these days, outside of the select few top research universities in the world, are obsolete before they begin. At least from what my friends in the field tell me.
Anecdata of one: I finished my PhD about 20 years ago in programming language theory. I created something innovative but not revolutionary. Given how slowly industry is catching up on my domain, it will probably take another 20-30 years before something similarly powerful makes it into an industrial programming language.
Counter-anecdata of one: On the other hand, one of the research teams of which I've been a member after my PhD was basically inventing Linux containers (in competition with other teams). Industry caught up pretty quickly on that. Still, academia arrived first.
I developed a new static analysis (a type system, to be precise) to guarantee statically that a concurrent/distributed system could fail gracefully in case of (D)DoS or other causes of resource exhaustion. Other people in that field developed comparable tools to statically guarantee algorithmic space or time complexity of implementations (including the good use of timeouts/resource sandboxes if necessary). Or type system-level segregation between any number of layers of classified/declassified information within a system. Or type systems to guarantee that binary (byte)code produced on a machine could find all its dependencies on another machine. Or type systems to prove that an algorithm was invariant with respect to all race conditions. Or to guarantee that a non-blocking algorithm always progresses. Or to detect deadlocks statically. etc.
All these things have been available in academia for a long time now. Even languages such as Rust or Scala, that offer cutting edge (for the industry) type systems, are mostly based on academic research from the 90s.
For comparison, garbage-collectors were invented in the 60s and were still considered novelties in the industry in the early 2000s.
Perhaps - but normally you'll have a narrowly defined and very specific technical topic/hypothesis that you're working on, and many/most of these aren't going to be closed off by ChatGPT4
Will this effect the job market (both academic and commercial) for these folks? It's very hard to say. Clearly lots of value will be generated by the new generation of models. There will be a lot of catchup and utilisation work where people will want to have models in house and with specific features that the hyperscale models don't have (for example constrained training sets). I'm wondering how many commercial illustrators have had their practices disrupted by Stable Diffusion? Will the same dynamics (what ever they are) apply for the use of LLM's?
> but normally you'll have a narrowly defined and very specific technical topic/hypothesis that you're working on, and many/most of these aren't going to be closed off by ChatGPT4
Pretty hard disagree. Even if your NLP PhD topic is looking at hypotheses on underlying processes about how languages work (and LLMs can't give you this insight), 9 times out of 10 it's with an eye for some sort of "applicability" of this for the future. GPT-4 just cut off the applicability parts of this for huge swaths of NLP research.
I'm not too worried about that. We don't actually understand fully how LLMs function internally, so research on how language works and how to process it is still useful in advancing our understanding. It may not lead to products that can compete with GPT, but PhDs aren't about commercialisation, they're about advancing human knowledge.
All these people don't understand how hireable and desirable they are now. They need to get out of academia and plugged into AI positions at tech companies and startups.
Their value just went up tremendously, even if their PhD thesis got cancelled.
Easily millionaires waiting to happen.
---
edit: Can't respond to child comment due to rate limit, so editing instead.
> That is not how it works at all.
Speak for yourself. I'm hiring folks off 4chan, and they're kicking ass with pytorch and can digest and author papers just fine.
People stopped caring about software engineering and data science degrees in the late 2010's.
People will stop caring about AI/ML PhDs as soon as the challenge to hire talent hits - and it will hit this year.
That is not how it works at all. You won't get hired if you don't have the academic pedigree in the first place. That means a completed Ph.D and good publications in good journals.
Sorry, that's patently untrue. Perhaps it's anecdotal, but I know a host of undergrads who got head hunted into quite elite tech positions either directly from Uni where I studied, or due to private projects they were in. And I even know a few that doesn't even have any uni edu that got hired to very high technical positions. Usually they were nerdy types who had worked with or had exposure to large systems for whatever reason, or who showed some promise due to previous work, demos or programs they'd made. But sure, most people have to go the edu route. It's the safest way into tech, as you are - at least in principle - fully vetted before you apply. Thinking that you can get a data science or hacker job just by installing Kali is ofc also very untrue.
I think my post is more representative of the truth than yours. I am sure you are telling the truth, but these unique talents you are talking about are not representative of the bulk of people working in research.
The demand for AI/ML will fast outstrip available talent. We'll be pulling students right out of undergrad if they can pass an interview.
I'm hiring folks off Reddit and 4chan that show an ability to futz with PyTorch and read papers.
Also, from your sibling comment:
> Maybe it is also a matter of location. I am in Germany.
Huge factor. US cares about getting work done and little else. Titles are honestly more trouble than they're worth and you sometimes see negative selection for them in software engineering. I suspect this will bleed over into AI/ML in ten years.
Work and getting it done is what matters. If someone has an aptitude for doing a task, it doesn't matter where it came from. If they can get along with your team, do the work, learn on the job and grow, bring them on.
I recommend taking all the introductory courses you can find on both AI and ML. If you like the introductory courses, and you feel compelled to move on, then chances are you'll do well in a job regarding AI or ML. There are also several ways into it, either through pure mathematics, statistical modelling, data science, particularly through learning about various algorithms and reading papers, or even through practical application within data warehousing or day-to-day programming. I'd say it helps to have an academic background in either IT, statistics or mathematics, though, but depending on what you're aiming for it doesn't need be a firm prerequisite. Btw. linguists or anyone interested in natural language ought also apply!
I guess we are living in two different universes. Any job ad for an ML role or ML adjacent role says Ph.d required or Ph.d preferable. Maybe it is also a matter of location. I am in Germany.
For a plain SWE role a Ph.d might be a disadvantage here too, but for anything ML related it is mandatory from what I can see.
In my hiring experience as an interviewer, 90% of candidates with PhD or not will actually have mediocre grasp on ML. It is a rare happy day when I get a good candidate. We interview for months for one hire. I got to interview candidates worldwide so I've seen people from many countries.
As someone who hired for this in general we'd use PhD (or maybe a Masters degree) as a filter by HR before I even saw them.
It's true that a PhD doesn't guarantee anything though. I once interviewed a candidate with 2 PhDs who couldn't explain the difference between regression and classification (which was sort of our "ok lets calm your nerves" question).
Yeah, you don't want to be anywhere near a place claiming to hire HS graduates/4chan posters in disciplines requiring advanced knowledge for successful product development, unless, idk, they have demonstrated mathematical talent through well-established media e.g. math olympiads, thesis on some relevant discipline.
Almost all the time, they're shitty startups, where bankruptcy is a matter of time, run by overpromising-underdelivering grifter CTOs pursuing a get-rich-quick scheme using whatever is trendy right now -crypto, AI, whatever has the most density on the frontpage-.
Yeah true, I've had to work with too many fresh college grads to not relate to this. People try to take some rare case and generalize when that's really not applicable.
As much as I'd wish to say "you're wrong, people care about intelligent, passionate people who do great work, not PhDs" you're right about much of the work out there.
We've tried many time to work with CSIRO (the NSF of Australia) and it's fallen flat. They love impressive resumes and nothing else. I'm having a chat with their "Director of ML" who's never heard of the words "word2vec" or "pytorch" before. (And I'm a UX designer!)
I think at most corporate firms you'll end up running into more resume stuffers than people who actually know how to use ML tools.
Chris Olah was at OpenAI but is now one of the founders of Anthropic but doesn't have any degree (he joined Google Brain after dropping out of his undergrad degree).
I think they are pretty exceptional though - most people on the ML team do AFAIK. Perhaps on the infrastructure side the backgrounds are different.
We are just building a chatbot to assist students that study (that study AI itself, to be precise).
We could not "just use ChatGPT" because the answers have to be correct, or students could take legal action against the university in situations where they'd give an answer in an exam that the chatbot gave them and that might be wrong.
It's exciting for sure to compare e.g. ChaptGPT with other approaches, for a set of controlled benchmark questions such as:
-- snip --
Q: What is the A* algorithm?
A: The A* algorithm is a popular heuristic search algorithm used in pathfinding and graph traversal. It is an extension of Dijkstra's algorithm, but uses a heuristic function to guide the search towards the goal node, which can make it more efficient than Dijkstra's algorithm in certain situations.
The algorithm works by maintaining a priority queue of nodes to be explored, where the priority of each node is the sum of its actual cost from the starting node (g-score) and an estimate of the cost to reach the goal node (h-score), based on a heuristic function. The heuristic function must be admissible, meaning it never overestimates the actual cost to the goal node.
At each step, the algorithm selects the node with the lowest priority from the queue and expands it, generating its neighboring nodes and computing their priority scores. The algorithm then adds these nodes to the queue if they have not been explored before or if a better path to them has been found.
If the goal node is reached, the algorithm terminates and returns the path from the starting node to the goal node. If the queue becomes empty before the goal node is reached, then there is no path from the starting node to the goal node.
Overall, the A* algorithm is a powerful and widely used algorithm that has many applications in fields such as robotics, video games, and logistics.
-- snip --
This is variation on the common "it has to be perfect" argument. I don't really buy it.
Right now, GPT 4 would earn a top 10% SAT score, implying that it can give very high quality answers on a range of scholarly topic. This output is basically free.
Nobody that's beholden to even mild economic pressures is going to pay for an expensively constructed 100% solution if they can have the 99% solution for free.
This is where it pays off to be researching something completely esoteric rather than something immediately applicable. I mostly scoffed at such research in the past, but now I see the value of it. The guy researching QML algorithms for NLP is not panicking yet, I think.
