Funny, that website ( https://qwen.ai/blog?id=qwen3.8-omni-flash ) downloads automatically 441 MiB of mp4 files... (using Firefox on Linux PC - noticed it because my network chart in Gkrellm spiked for several seconds).
I know nothing about "vLLM" -> anything better compared to "llama.cpp"?
I started with "Ollama" (precompiled version) and it worked and was good enough to understand the very basics.
Then I downloaded the sourcecode of "llama.cpp", compiled it with specific compilation options for my GPUs (CUDA/nVidia using proprietary module on Gentoo Linux) & CPU (AMD), and the same model ran twice as fast -> since then I stuck with "llama.cpp" (and "ik_llama.cpp" in very few cases).
I honestly don't know what made "Ollama" (precompiled) so much slower than "llama.cpp" (compiled locally) at that time and I'm too lazy to doublecheck now, in any case I now absolutely love all the knobs that "llama.cpp" has to tune your hardware setup & your workload, which is the reason why I recommend it.
The parameter "reasoning_effort" is something new, or am I wrong?
Qwen3.8 comes with official support for reasoning_effort, which can be used to adjust reasoning depth and control cost:
- xhigh (default): for complex tasks demanding thorough analysis
- medium: balancing accuracy and speed
- low: efficient reasoning optimizing for speed and cost
In addition, preserve_thinking is enabled by default for all workloads for the best out-of-the-box experience.
Asking because in my case (OCR of scanned historical "National Geographic" magazines) the LLM trying to merge text split into separate columns was running in circles from time to time and needed a lot of prompt tuning when using Qwen 3.0/3.5/3.6 (still needs from time to time).
I'm using Qwen 3.5 for OCR, and reasoning_effort is supported there too. I found that it can be loop-prone (though somewhat less so) even if you set reasoning_effort to low.
Also an important part that keeps getting left out for some reason: they're doing so illegally. They are running unpermitted turbines. They claimed the turbines were off and journalists had to use thermal cameras mounted on drones to prove that they are indeed running. Then a non-profit had to sue them to get them to agree to turn off the turbines by 2027...
That's how bad our system is at giving any consequences to white collar crime.
and they (and many other AI data centers) run on some of the environmental/health hazard worst kinds of gas turbines available (i.e. cheap portable ones)
A well build gas power plane can be one of the cleanest fossile fuel power plants with neither huge issues of air nor sound pollution (compared to other fossile fuel, it still isn't perfect by far)
Same btw. for data center cooling, my office was for 3 years above a data center of a super computer in the mittle of a city and there where no sound issues at all (through yes it was a special case in many ways).
I mean yes, it costs more and many well established companies for high quality gas turbines are booked out.
Still using the kind of gas turbines which can bring back literal acid rain shouldn't be an option at all.
Lets also not speak about the environmental/health damage the production of the natural gas through fracking does.
Which bring us to a common nonsense argument, people going on and on about hoe economically expensive/harmful climate protection is, but totally missing that many oft he circumvented laws are not for climate protection but are for environmental protection for the sake of humans health.
To make thinks worse most of this data centers seems to be more about making money, then being in any way relevant for winning any AI race...
Problem is all the (relatively) efficient and (relatively) quiet turbines have long since been snapped up, so the AI sloppers have to go with inefficient, noisy ones for the foreseeable future. And given that no-one knows when the AI bubble will finally burst, no manufacturer wants to sink hundreds of millions into new plant to produce turbines in a year or two when the sloppers may be barely alive on taxpayer cash injections.
No, there are extreme consequences only when toes are stepped on. For example SBF... If it was run by Musk or Trump you can be sure there would be no sentence. I'm not arguing what SBF did isn't a crime. I'm saying if that deserves 20 years, then everything else should also be equally punished.
It’s an industrial supply bottleneck and rollout timeline incompatibility rather than a Dr Doom Evil Genius Complex that prevents everyone from deploying solar. The race dynamics make the discount rate on GPU deployments very high.
Solar is preferable when you have a ton of unproductive land and longer timelines to set it up - it’s certainly going to be cheaper than gas when everyone goes sets up NG plants as option 1 for new energy capacity.
The other confound is that China is the primary supplier of battery and solar tech, I’m sure that supply chain risk becomes part of the calculus when setting up so much infrastructure.
For scale, the current DC is about 8k acres, and the solar to run 7.6GW is almost 40k acres.
cheap, dirty, loud "tbh. shouldn't exist" kind fo gas generators are far more available and what is widely used
but also huge part of the solar supply bottleneck is artificial due to import regulations from china and both the US and EU having failed to secure meaningful domestic solar supply chains.
»It’s an industrial supply bottleneck and rollout timeline incompatibility rather than a Dr Doom Evil Genius Complex that prevents everyone from deploying solar.«
- Running billions of dollars worth of GPUs only 30-50% of the time insanely wasteful. Everyone else is running them at 99% duty cycle or close to it.
- Solar is big and generally has to be built far from where the data centres are. Transmission is already a bottleneck in many states. Adding tens of gigawatts requires non-trivial large scale infrastructure investment by… the utility companies, not SpaceX. Not to mention regulatory hurdles, etc.
- Battery production is already mostly maxed out, so obtaining tens of gigawatt hours of capacity on short notice is a decent percentage of global production.
Gas and other fossil fuel power plants can be deployed adjacent to the data centres for effectively zero transmission loss and no long powerlines that upset the NIMBY types.
They can work 24/7 and are largely available right now, but as others have pointed out the turbine blades are a limiting factor with these too.
> Solar is big and generally has to be built far from where the data centres are.
these data centers dont care that much about network throughput - you can sneakernet the training set into place and inference on the text isnt that taxing on the network
there's a reason the ai datacenters are being built wherever the power is cheapest and theres the lowest regulations. there's no real physical location requirement, so you could actually put them close to the solar power generation
Solar works as well as it does only because we’ve spread it out and connected it to a grid. When it is overcast in one spot it is sunny elsewhere.
That “elsewhere” has to be far enough away to get different weather.
This means a grid capable of transferring 100% of the needed power.
So even if you plopped a new DC in the middle of a field of panels you’d still need a ~1 GA grid connection… or a battery array many times the size as you’d otherwise need just to cover night time usage.
All those companies are sitting on literally hundreds of billions of dollars in cash. If they need this so badly, let them build the connection and other infrastructure.
They want to build the data centres right now, but the power companies and other energy suppliers are booked solid for 5+ years out.
Money or not, you can't just click your fingers and instantly get regulatory approval, or spin up a facility to build parts that only a handful of places have ever figured out how to mass-manufacture.
Gas turbine blades are especially difficult, but batteries and panels are not far behind. There are a surprisingly small number of shops capable of making power switching gear and transformers too.
Idk. about any specific data center but many of this data centers aren't for AI training / making advancements on the AI race. They have sufficient resources for that already AFIK.
They are to run AI, so that the 100th app can force an AI agent down your throat weather your want or not /s. (Or in more optimistic words, so that the AI adoption can spread as fast as their is demand.)
The problem is that the grid can’t get built out fast enough. You’ll probably see most AI data centers switch to grid power in the coming years using mostly wind and solar because it’s simply cheaper as well
They could (and should be required to) build out their own solar/wind/battery infra as part of standing up a new datacenter. So much money is being wasted on poisoning our air and further enriching the richest companies in the history of the world.
Remember when they said they couldnzt electrify the grid for ev's so they opted to just not do anything with ev's. Now datacenters are getting rushed in so fast and breaking the grids so hard they can illegally running generators
You totally can. I've seen cases a rat gnawed a circuit breaker and blacked out a DC. Fires too. It's not an industrial manufacturing facility. Computers can just be shut off and on. There's also nothing needed to bootstrap the other parts like cooling etc.
A rat taking out a DC resulting in a black start event isn't the same thing as being able to turn a DC on and off at will. The thousands of computers in an DC running various bits of software don't just power on like your Windows computer and result in a working data center after a couple of minutes automatically.
Are the UPSes actual batteries or just backup generators? If they're generators then that defeats the point. If they're batteries then it's kinda just regular solar then?
But that's the point. It's not like a solar DC would suddenly lose power because of cloud cover. Large scale batteries can provide amble ride through inclement weather to allow a graceful shutdown, and weather forecasts at that scale are accurate enough to predictively model your energy availability.
AI training is unique because it's not interactive workload serving.
You’re talking about one of the most hated person on hearth with one of the worst reputation. The literal Nazi salute guy. I don’t think he cares much about positive PR like that
Renewable energy only makes sense in a society that is agrarian. The second you have to make anything industrial you need baseload. That baseload can either be coal or nuclear.
In modern times you combine solar energy with wind power and batteries and you already have 90% of the load covered. Then combine that with cheap, easily adjustable gas power plants to complete the picture for cloudy, windless winter days.
There's really no need for coal or nuclear power plants anymore.
Building gas power plants that run only a few weeks per year is very expensive. Just compare prices in Germany (gas) with China (coal) or Finland (nuclear).
A more economical way would be to build nuclear power plants that power electrolyzers most of the time and provide power to the grid when necessary.
In the US, gas power plants make more sense due to ridiculously cheap gas, but even then the capacity factor of those power plants is much higher than 10%.
Comparing 24/7 AI training to operating hospitals 24/7 is a bit of a false equivalency, don't you think?
Besides, hydroelectric is commonly used as baseload where available. Additionally, there are big strides happening in power storage options to smooth the effective baseload of solar and wind.
I would argue that coal energy only makes sense in an anachronistic society doomed to fail.
You've been confidently wrong a couple times now - or maybe intentionally misrepresenting the facts?
In the US there are several states where hydro is the majority source of baseload electricity: Washington, Idaho, Vermont, Oregon, etc.
Outside of the US there are several countries that source more than 90% of their electricity from hydro - at higher percentages than Norway even: Paraguay, Albania, Democratic Republic of Congo, Nepal, Namibia, Zambia, Tajikistan.
You also don't have to build mountains to utilize hydro. There are areas that are utilizing pumped hydro to store the kinetic energy to smooth the baseline.
I've worked in energy and ran the numbers. Renewables only make sense for a pre industrial or deindustrializing society. When you need power to make shit they are useless. Which is why the industrial revolution moved from water to coal in the first place.
Yep and China is leading the world in recent and active construction of traditional hydro (dams) and pumped hydro storage to reduce their reliance on coal for baseload. I'm done with your bad-faith arguments.
Their 30 year plan is to move to nuclear for base load, and they are on track to manage it. The only people who think China is moving to a pre 2020 like western grid are people who don't know the first thing about China.
Or we can turn off the net negative data centers in order to keep our Hospitals running smoothly, rather than taking the entire grid down so some psychopaths in Silicon Valley can LARP at creating their silicon God.
I use it since ~20 years on most of my notebooks/PCs/root servers mainly because of its flexibility and I often end up wanting to use the very latest versions of software (e.g. ZFS, kernels, etc...) without struggling as most things are available from the official central package repository. I admit that updating/upgrading the OS is a pain from the time perspective (because of the long compilation times). As a programmer/developer the experience is as well fantastic as most of the libraries, compilers, interpreters, etc are very well integrated or easily installable.
I did try to use a few times Linux Arch, but I never fully liked its AUR.
For VMs I basically use always either Debian or Linux Mint with Xfce.
For gaming, mediacenter-PC and webradio-miniPC I use Linux Mint with Xfce.
It's funny that whenever I need assistance with some task (fixing a problem, doing something weird, ...) and I mention to Google Gemini something like "I'm using Gentoo Linux blahblah" then often Gemini starts its reply with something like "Ah, a picky Gentoorian, I see - then let's do this the complicated way..." :o)
For the ones that don't know "nmon", have a look at it as well! (press "h" to see the list of available monitors - press it again to make it go away, press "q" to quit)
> I have trouble getting my computers to copy data between each other anywhere near the theoretical limit for a 1 Gbps NIC, despite being connected through only a single switch.
Maybe your LAN cables have deteriorated?
Reaching 100 MiB/s should be easy peasy for a PC; in my case (2 10Gbps switches) by using nvme SSDs I reach almost 950 MiB/s and with 4x raidz1 HDDs I reach 450-650 MiB/s.
Until a few years ago I was smoking in my flat and often I had to change lan cables (whatever's in the smoke was sticking to the connectors)
reply