brucethemoose

https://lemmy.world/u/brucethemoose

Public Key

npub1v7cwgcnn0u5xvdcvyfvjw5crm3fuu25jmkgx0kxegqpwy9tau2lse65daj

NIP-05 Address

brucethemoose_at_lemmy.world@momostr.pink

Profile Code

nprofile1qqsx0v8yvfeh72rxxuxzykf82vpac57w92fdmyr8mrv5qqhzz47790cprpmhxue69uhhyetvv9ujumt0d4hhxarj9ecxjmntfpwl0g

Publishing to

relay.momostr.pink

Author Public Key

npub1v7cwgcnn0u5xvdcvyfvjw5crm3fuu25jmkgx0kxegqpwy9tau2lse65daj

Show more details

Last Notes

2026-07-30 22:45:19 UTC

- reply

Also, to add:
Information hygiene here is bad.
Every day, nearly every time I scroll the feed, some spicy headline goes past. When I go to look it up, it’s often completely made-up misinformation, or a warped ragebait headline, or a literally bot-written Tweet, yet it’s voted right to the front page.
…I thought Reddit refugees would be sick of this, and more careful/skeptical, but apparently not. I’ve even talked to some users who don’t understand the distinction between opinion columns and news.
And I had an argument with a mod, who had no problem with an LLM engagement farm Tweet as long as its “the right message.”

2026-06-29 01:59:04 UTC

- reply

I have a single 3090!
That's the dream GPU, these days.
And I have 128GB CPU RAM. So the best model I can run is MiMo 2.5 (a 300B model) at around 10 tokens/sec, using hybrid CPU inference.
...But that's the worst-case scenario, for speed. It's an IQ3_KT quant (a high quality "trellis" quantization type, but very slow on CPU), with a gigantic model that barely fits in my RAM+VRAM combined, with no DFlash or any kind of speculative decoding turned on. I could tune it to be much faster, but I mostly just want "max quality, fast enough to read as it streams, barely fits in memory" for this model.
For speed, or prompts with lots of thinking or context (like agenic use), I just run Qwen 3.6 27B now. That would fit in your 3090 no matter how much CPU RAM you have, but you have to be smart about the backend and quantization you pick. If you just use Ollama, it's gonna tell you it won't fit, or use some horrible default that spits out garbage.
***
...This is what I meant to emphasize.
It's not just the hardware. You kinda have to be part developer, part enthusiast to even follow this stuff, it up optimally, and keep it up-to-date. If you just try to Google "best LLM for 3090," you will get *absolute garbage*.

2026-06-28 22:10:35 UTC

- reply

You don’t even need Claude anymore. GLM 5.2 API is good enough for 95% of the same things and vastly cheaper.
MiMo 2.5 Pro and Kimi are also very good. And then there’s Cerebras API if you just want simple things done *quick*.
The thing with self hosting, while awesome, is that it requires a lot of hardware and considerable time investment for what’s essentially a “base tier model,” or at best one step down for what’s still a very cheap API. I still love it, especially the privacy and control aspect, but you aren’t running Claude at home unless you’ve got a threadripper or server hardware collecting dust.
…Hence I can understand why people don’t pursue it. Especially since a cursory Google search will lead you to trying the Deepseek distillation on Ollama (which is awful).

2026-06-27 20:56:22 UTC

- reply

I'm not consistent about it yet, but because of *exactly* this, I'm trying to differentiate the two when I talk.
Responsible automation? I use ML or machine learning.
The grift consuming the world? A Tech Bro? "AI"
I think one of the saddest things is the conflation between the two, like you can't even talk about one without invoking the other. Or it opening up that whole ethical debate, when you're just talking about, like, a 100M transcription model trained by one research in some university on a potato.

2026-05-06 14:30:35 UTC

- reply

Okay, I fudged the part about "for free." The problem is DeepSeekv4 is literally in preview, and its architecture is so new that engine support for its weights is poor.
*Right this second*, you can either pay a few cents to try it from some API (there are many providers since its open weights), or rent a GPU (or maybe a CPU) instance if you don't trust the public tests, and actually want to test resource usage yourself.
Or you can quantize it and self host it. I plan to do so on my 128GB RAM/RTX 3090 desktop, which is a affordable config to rent if you don't have a desktop like that.
But llama.cpp support is a work-in-progress. Same with other backends like Ktransformers. Realistically your options are:
- Wait a week, maybe a few weeks, for the llama.cpp/ik_llama.cpp developers to implement to DSV4 architecture.
- Try one of the janky GPU/Apple forks availible right now.
- Try one of the slightly-less-janky, but slow CPU-only chinese forks.
But once its implemented, I'm going to make my own personal IQ3_KS mixed quantization for 128G desktops, and see how it compares to older architectures myself.
***
Another confounding factor is, if you're researching "AI farm inference costs," thats very different.
Frugal providers like Deepseek use complicated schemes to batch requests over many GPUs, with each taking requests in parallel. In other words, the more GPUs they have, the more *speed per GPU* they can squeeze out. For DeepseekV3, last I heard, Around 300 GPUs or so was an ideal deployment number...
And they aren't even going to be using Nvidia GPUs anyway. I believe Deepseek is switching to Huawei for inference.
But however you slice it, they're using order of magnitudes fewer resources than Tech Bro providers like OpenAI or Grok. They *have been*, for over a year.

2026-05-06 13:54:10 UTC

- reply

Also, on Deepseek V4… you can run it yourself, for free. There’s no mystery. And there’s tons of benchmarks out there already.
It’s indeed very efficient if you’re into long context. But at shorter context lengths, it’s not *too* different than Deepseek’s previous releases (and the flood of MoE models that have come since then).

2026-03-24 22:36:33 UTC

- reply

The source more succinct than the talking YouTube head, in this case. From the archinstall maintainer:
>I appreciate the work ahead of time, and the law is the law. @svartkanin raised this PR internally within staff channels, and the feedback is that we'll wait until there's an overall stance from Arch Linux on this before merging this, and preferably involve legal representatives on this matter on what the best way forward is for us.
> But from a personal reflection it's clear that there's a disconnect between law makers intent and how things like this will be implemented in reality, and once a law is in place - we might have to implement inconvenient things..
> So I'll leave this open for now, but I'll also lock the conversation because experience from the mailing lists on this topic has told us this thread will get out of hand quickly.
> @dylanmtaylor: this stance does not mean that we won't merge this. And despite locking this thread, I think you, me and other contributors and maintainers can still comment (which is fine, and good).

2025-12-04 18:12:02 UTC

- reply

I buy this.
I think publishers looked at Fortnite, Overwatch, Call of Duty, PubG, GTA *on a superficial level*, and concluded that their customers don’t care about *comprehensive* immersion. Gamers (presumably) like “serious” yet shallow, not quite tongue-in-cheek and not requiring much attention span.
I’m not sure if that’s the bathos you describe, but it seems close.
Hence publishers tried to reconcile that “Fortnite style” (an expansive, flashy veneer of lore, art, and cheese without anything beyond that) with traditional RPGs, and ended up “neither here nor there.”

2025-11-17 21:27:15 UTC

- reply

Have not played it yet, but for me, personally, E33 looks like one of those "better to watch the cutscenes on YouTube" games.

2025-09-10 14:35:35 UTC

- reply

If it were a nice person inside and they just liked the 'lifted' style though, that's one thing. I think there's something to be said for auto personalization culture, like people do with jeeps and the Slate is trying to start up. I actually knew a really short girl who drove a lifted truck and simply thought it was neat, heh.
But usually, I wince in anticipation of what flavor of 'macho' is going to step out. Like you said perfectly, it's usually not Hank Hill.

2025-09-10 13:12:26 UTC

- reply

Speaking as a Texan, I associate the worst of truck driving with “being an ass.” It’s not always true, but man, if someone’s gonna cut me off just to be a jerk, it’s usually a lifted abomination.
This though:
![1993 Ford Ranger](https://www.ranger5g.com/forum/attachments/hank-hill-jpg.152132/)
This is fine.
No, it’s magnificent.

2025-08-04 16:37:41 UTC

- reply

ODST was lovely. Halo needs “side stories” like that.

2025-08-04 13:30:22 UTC

- reply

Yes exactly! Same with the characters.

2025-08-04 11:32:11 UTC

- reply

4/5 made *so much* accumulated story baggage though.
Infinite would have been better if 4/5 and whatever requisite novels didn’t exist.

2025-07-20 20:25:47 UTC

- reply

Also, aren’t most folks using apps these days? I have elders *and* younger relatives that literally don’t know how to use a web browser.
I wouldn’t want to be a web publisher right now…

2025-06-16 17:12:17 UTC

Video is linked. SFW, but keep your volume down.

2025-02-14 22:10:08 UTC

- reply

That is not what I asked.
OK, let’s reverse it, and say Mexico renamed the Gulf to something else. What do you think Google should do, inside Mexico and out?

2025-02-14 21:51:18 UTC

- reply

Then why add “Gulf of America” in countries that do not recognize the name?

2025-02-14 08:46:52 UTC

- reply

You think Google and Apple should call it the Gulf of America?

2025-02-14 00:10:32 UTC

- reply

I can see Google (and Apple) quietly paying a fine and changing it for Mexico (and any other jurisdiction that specifically asks).
This is stupid and cowardly of them, but that aside, the implication is 100% “comply, or the US government will make life difficult for Google.” I *can see* how they’re stuck between a rock and a hard place.

2024-12-28 16:33:01 UTC

- reply

James Cameron’s Avatar series.
Then again… Does *anyone* actually like it? It seems to have all this online hype when it’s such a boring visual spectacle.
It’s like the opposite of the *other* Avatar franchise, which wasn’t a commercial hit, and seems less popular on paper, but seems to have a massive cultural impact.