Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

"I think that is not merited. Deep neural models have their own set of problems,"

But I think that is the issue...going forward wouldn't you hire a machine learning specialist rather than an NLP specialist for those problems? As far industry goes, is there any value in all the syntax/semantics/phonology theory NLP folks command?



I don't think so. In practice, you'll need to hire _both_ the domain expert and the ML specialist. Or maybe even no change at all... you still want the domain expert, because the problems may be fundamentally related to the framing of the task the AI system is trying to solve, not the model architecture or training/fine-tuning.

You definitely see this in the weather space. Despite flashy headlines, AI has really failed to make much of a difference at core weather forecasting, because the specialized statistical systems that combine many numerical weather prediction models are so greatly refined to the generic forecasting problem that there is little room for improvement. And AI practitioners rarely even focus on the actual interesting problems in the field where we suspect there can be huge gains - like convective initiation (predicting where exactly storms will form and their potential phase trajectory, e.g. what is the probably it will go tornadic or produce large hail?). The reality is that meteorologists can refine the prediction task so precisely that you don't need innovative, brand new model architectures. And the crazy brand new pure DL/data-driven models like NVIDIA's FourCastNet or DeepMind's GraphCast have a long way to go to be a practical competitor to traditional NWP and basic post-processing/statistical bias correction.


The late Fred Jelinek, founder and manager of IBM's speech recognition R&D team at TJ Watson Research Center @ Yorktown Heights created the famous joke "every time I fire a linguist, recognition accuracy goes up" - as someone with a stake in both ML and NLP I would say more credit goes to ML than to NLP for sure.

Will linguistic knowledge not be needed at all? I don't want to speculate about the far-out future, but what I can safely say from industry experience that at any stage (1996 - now) there was always some extra gain to be had on top of the ((statistical | neural) - only) approach of the day by engineering hybrid solutions that at some level also exploit human-injected linguistic knowledge and human-injected business rules.

Next month at ECIR 2023 in Dublin, I will present a "shoot out" between a BERT model especially pre-trained (for months) and fine-tuned for document summarization of financial meetings (earnings calls) and a one-line POSIX shell script (two cascaded grep commands) written in 3 minutes by yours truly that also extrcts a summary - with surprising results...




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: