I've spent 10 years implementing new technology into businesses from all over the world and the thing that has killed the most projects at the proof of concept phase has not been whether it works. It's whether the cost of going to production from PoC produces a significant ROI. AI is absolutely changing the economics there.
Humans were simply nowhere near answering all the useful questions they had which software could answer. The problem was that software was expensive.
Now, software is cheaper and so people who couldn't afford to answer questions they've long wondered about can afford to answer them.
>people who couldn't afford to answer questions they've long wondered about can afford to answer them.
If the software isn't downright free – today, you can slap a $349 RTX 5060 [†] into any POS surplus computer and have an offline assistant, running Llama or Ornith, via Ollama in linux (i.e. the LLM and OS are FREE open-source software) [∑]
I have a working demo of this that has been shown/leant to several friends, and they're all surprised that it "all works so well, without being online, for only a few hundred dollars."
When I've demonstrated Mistral-small on my 5070Ti... that has been a real jaw dropper. I haven't shown anybody qwen3.8:27b, yet... but dammmmm, what a month of learning it's been.
----
A friend that was going through wifecancer confided in me that "you can ask it anything, without feeling embarassed" – and that has stuck with me (that so many smart people are afraid to ask simple questions [*], out of perceptional worries).
----
[†] (8GB DDR7) brand new from Wal-Mart
[∑] My first linux/LLM machine was built with setup help from Perplexity.ai (with a dozen "assists" - I am bluecollar, non-coder). This used an obsolete i5 (and $200 used VEGA64 GPU) to create a decent Llama3.1 LLM machine (~100wpm typeback, perhaps 70 tokens/s).
[*] even to their own detriment, of shame, when simple solutions often do exist
No longer convinced that’s true. Some of the indebted providers might go under but there is nothing preventing someone from just setting up a new provider and serving tokens debt free. GLM or whatever.
That provides continuous downward pressure on token prices even if it isn’t Astra level
Agree model providers can't take much margin, a few percent plus maybe a bit extra from those willing to pay for a "better" model. Upstream is more concentrated: TSMC, NVIDIA, AMD could maybe raise prices and capture more of the value, which would affect open model providers.
Do you have any evidence to suggest that this true? And if so, why can't I just use any of the very good open source Chinese models (and eventually American once reflection, thinking machines, etc catch up).
Exactly. You can buy on server grade mobo and CPU, stock it with ddr4 and run qwen and basically get gpt 3+ level quality now. A year from now, I imagine we'll see even better efficiency.
I think this is a really sober comment. There are lots of knock-on effects of this claim, even if it's not true which are consequential. The fact that a spokesperson for the US government is going out of their way to comment is concerning.
It's especially sad because on of the things those LLMs are best, almost purpose built for is to tell those students which books to go open, where to find nuggets people haven't bumped into for years, to make cross-connections that would take a PhD a decade to find.
My bona-fides here amount to little more than being a big time reader of non-fiction. But IMO, there's a nice synthesis here. Rather than going long rounds of asking LLMs about a subject, I usually end up asking it for book recommendations. It's much better than a google search and you can push it into some deep corners if you go past the surface level recs.
You can get quite specific. You can find texts you wouldn't discover unless you spent years studying the topic. Often these are completely approachable and give interesting perspectives they just get buried behind a wall of syllabi and listicles.
This is how I ended up reading Thompson's 'The Making of the English Working Class' and Graves' 'Goodbye to all That' among others.
Yeah, this is very much one of those stories where people from many different perspectives or chopping it up on a plate and ripping it through a straw.
I think the important lesson is to use clear eyes to evaluate what the rewrite buys you. I was on a team that rewrote a native code app in C#. We also had access to early cloud tech in the Azure stack, what is called queue now and then was called service bus.
These two technologies combined greatly simplified this specific product making it far easier to maintain. Performance on these services was not important so native code was carrying a lot of penalties without the benefits.
Having a well documented messenger like service bus with great SLAs removed several tools we had needed in the old implementation.
We were able to leverage the tests form the original product to define success and tmthus were able to solve a lot of the edge cases in the new code w before we even shipped.
However, the old code was perfectly fine code. If new technologies had not provided significant simplification of the service architecture, a rewrite would've been foolish. And without the very good previously existing tests, we would've run into a lot of issues as we released.
What do your tests look like. Because rewriting by hand and rewriting via AI have the same load bearing on whether or not your tests cover your scenarios and your integrations well.
Because there is more value in understanding someone else's ideas than as some kind of cultural favor? East Asians built empires, invented and discovered incredible things. They have developed elaborate artistic, musical and familial traditions. All of that is of course related to their cosmological ideas. If you could, why wouldn't you understand ideas that were integral to so much human activity? And if they fail to understand your culture in the same regard, that just puts you at an advantage.
In the words of my old man, "I'm not telling you what I know, cause then you'll know what I know plus what you know and then you'll know more than me!"
Humans were simply nowhere near answering all the useful questions they had which software could answer. The problem was that software was expensive.
Now, software is cheaper and so people who couldn't afford to answer questions they've long wondered about can afford to answer them.
reply