Nostr notes by Al Sutton

Small data point for folk who are playing with the #Mistral Nemo ...

2025-06-03T09:56:14Z

Small data point for folk who are playing with the #Mistral Nemo 12b Q4 #AI model.

We're now using it for code reviews at Snapp Automotive (npub1hnk…syy4), and an m4 mac mini is only 30-40% slower than a 22 core, 256GB Ram, RTX 4000 Ada workstation.

A suitable m4 mac mini costs about half of what the RTX 4000 ada GPU card alone costs.

Mac Mini's seem to be a decent way to get into playing with #LLM usage.

I think I’ve answered my own question about whether things ...

2025-06-02T15:05:03Z

I think I’ve answered my own question about whether things could go meaningfully viral on #Mastodon.

I’m sure @npub12yu…k2zm would love another PR from an ...

2025-05-30T18:24:20Z

In reply to nevent1q…thwm
_________________________

I’m sure Erik HELLman (npub12yu…k2zm) would love another PR from an experienced contributor :)

I’m playing with a few, but at the moment I’m preferring qwen ...

2025-05-30T18:18:25Z

In reply to nevent1q…5azu
_________________________

I’m playing with a few, but at the moment I’m preferring qwen coder 2.5 with 7b params at Q4. I’ve found some larger models generate more wordy responses, but qwen coder 2.5 was more focused on actual issues rather than vague concerns.

Here’s the Action we’re using; https://github.com/ErikHellman/claude-code-reviewer/pull/1

Yup. The CI server is self hosted, and runs ollama locally with ...

2025-05-30T15:54:57Z

In reply to nevent1q…vf07
_________________________

Yup. The CI server is self hosted, and runs ollama locally with the appropriate model available.

Today I deployed an #AI backed code review system as part of the ...

2025-05-30T15:04:32Z

Today I deployed an #AI backed code review system as part of the Snapp Automotive (npub1hnk…syy4) #monorepo PR review checks.

Is it going to replace coders? No. Is it throwing up amazing insights? No.

Is it going to catch small things before PRs get opened for human review? Probably.... and that's where I see value. The more things an automated system catches, the fewer things humans have to spend time looking at.

#Dev #DevEx #Coding

“What if we are the monster?” was such a twist from Rosy. ...

2025-05-30T08:15:35Z

In reply to nevent1q…edap
_________________________

“What if we are the monster?” was such a twist from Rosy. 😮

Good advice, I’d expect anyone wanting truly private ...

2025-05-05T06:25:31Z

In reply to nevent1q…etag
_________________________

Good advice, I’d expect anyone wanting truly private conversations to use a system with E2E encryption, hence my question earlier in the thread.

Bit of a repurposing solution, and I know you’ve had a rough ...

2025-05-04T09:32:20Z

In reply to nevent1q…usxj
_________________________

Bit of a repurposing solution, and I know you’ve had a rough time without it recently, but what about Mastodon private, targeted, messages?

Cool. In terms of IM, my main takeaway from being the TL on the ...

2025-05-04T09:03:59Z

In reply to nevent1q…3g0t
_________________________

Cool. In terms of IM, my main takeaway from being the TL on the Facebook Secret Messages team many years ago, was that browsers are a pretty insecure platform if you’re trying to keep message histories secret due to the access some browser plugins have.

Do you want E2E encryption, or are you happy without it?

I find https://european-alternatives.eu/categories useful for ...

2025-05-04T08:49:15Z

In reply to nevent1q…7xha
_________________________

I find https://european-alternatives.eu/categories useful for suggestions, and then trying things out to see what works best for me.

https://en.m.wikipedia.org/wiki/Trooping_the_Colour

2025-05-03T06:20:54Z

In reply to nevent1q…zqzq
_________________________

https://en.m.wikipedia.org/wiki/Trooping_the_Colour

If you vaguely thought the #UK government understood tech… ...

2025-05-02T16:23:29Z

If you vaguely thought the #UK government understood tech… https://uk.lemmy.zip

There are many folk talking about today being the hottest start ...

2025-05-01T05:54:55Z

There are many folk talking about today being the hottest start to May for the #UK since records began.

It's also worth remembering that "Net Zero" doesn't been back to previous levels, it just means we're not making things worse.

In the #UK the "Net Zero" target is 2050.

That's 25 years of things still getting worse.

25 years.

😬

https://www.bbc.co.uk/news/live/cwy0mwvx04lt

#Climate #ClimateChange

3 is the best approach I’ve seen used anywhere. I think your ...

2025-04-30T08:30:40Z

In reply to nevent1q…s6ld
_________________________

3 is the best approach I’ve seen used anywhere. I think your choice of trade-offs is good.

If anyone else encounters this, it’s because you have the wrong ...

2025-04-28T14:18:46Z

In reply to nevent1q…pc44
_________________________

If anyone else encounters this, it’s because you have the wrong IP address for the server being given to your client 🤦‍♂️ #Debian12 #Debian #OpenSSL

Today keeps getting stranger... next on the list, #Google pushing ...

2025-04-28T13:31:45Z

Today keeps getting stranger... next on the list, #Google pushing to the #AOSP some repo manifests which require Google Internal SSO authentication;

https://android.googlesource.com/platform/manifest/+/refs/tags/ub-automotive-master-20250418/aosp-master-with-modules.xml

Oh this has unexpected fun written all over it.... ...

2025-04-28T12:15:38Z

Oh this has unexpected fun written all over it....

alsutton@t5810:~$ openssl s_client -connect downloads.snappautomotive.com:443 -servername downloads.snappautomotive.com
CONNECTED(00000003)
401729AACC7F0000:error:0A000438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error:../ssl/record/rec_layer_s3.c:1605:SSL alert number 80

#Debian12 #Debian #OpenSSL

A couple of weeks after sorting out the quota issue on #Ionos ...

2025-04-28T11:58:13Z

A couple of weeks after sorting out the quota issue on #Ionos (50GB hard quota on a 250GB package), but them manually increasing the quota to be, you know, actually 250GB, guess what's happened?

Yup. #Ionos's system has reverted the quota back to 50GB and I'm seeing failed upload messages again.

Time to switch to Hetzner (npub18s7…wqln)

I may get some time to write a GitHub/Forgejo action which we can ...

2025-04-25T11:03:57Z

In reply to nevent1q…vr69
_________________________

I may get some time to write a GitHub/Forgejo action which we can run on a daily basis and to get regular update PRs.

Thanks to @npub1ang…ahch's #gradle version catalogue update ...

2025-04-25T08:03:02Z

Thanks to Hugo (npub1ang…ahch)'s #gradle version catalogue update plugin, I now have a replacement for #GitHub #Dependatbot, which means we're on feature parity for our #Forgejo instance 🕺 .

Thanks Hugo 🤗

https://github.com/littlerobots/version-catalog-update-plugin

Whats the EU/UK alternative to AWS’s range of services?

2025-04-12T11:10:57Z

In reply to nevent1q…6rln
_________________________

Whats the EU/UK alternative to AWS’s range of services?

on-prem prices can go all over the place. I use workstation cards ...

2025-04-08T08:54:57Z

In reply to nevent1q…v088
_________________________

on-prem prices can go all over the place. I use workstation cards for local LLMs, and the price of an RTX 4000 (Turing generation) is pretty low.

If you're thinking of buying for on-prem then evaluating Ada Generation cards are worth it due to native support for more formats. The only down side is the stupid-ass power connector on anything higher than a RTX 2000 Ada.

A 4b parameter model at q4 is a pretty small model (~3.3GB), so ...

2025-04-08T08:52:19Z

In reply to nevent1q…mfjg
_________________________

A 4b parameter model at q4 is a pretty small model (~3.3GB), so you're not really pushing limits on either.

At that level you might be seeing some accuracy issues that will need manual intervention, so going Q8 before going for higher parameter counts could be useful.

If you want to see where things differ then the 12b model will start to show things up. The CR instance could even handle 27b @ Q4, which I would expect the mac to struggle with.

To me the test is "Is it generating faster than I can make ...

2025-04-08T08:23:14Z

In reply to nevent1q…j3p6
_________________________

To me the test is "Is it generating faster than I can make use of the information (e.g. read it)?".

If you want to speed things up try an 8 bit quantized model (Q8), so that the weights are all 8 bit ints. The Cloud Run GPUs are Nvidia L4's, so if you're using the full precision (FP32, i.e. 32 bit floating point weights), you can speed things up by going to FP16 or even FP8.

The other thing with CR is making sure the model fits into the GPU RAM and doesn't overflow into system RAM.

TIL: There is a category of goods in the #UK called "80% ...

2025-04-03T16:40:02Z

TIL: There is a category of goods in the #UK called "80% Turkeys" :O

https://assets.publishing.service.gov.uk/media/67ee668c9eae202448299c5b/request-for-input-indicative-product-list.pdf

Moving our git repositories off #GitHub to #Forgejo is proving ...

2025-04-03T15:47:56Z

Moving our git repositories off #GitHub to #Forgejo is proving interesting.

The migration process Forgejo has built-in is really good, but, unfortunately, action runners have a few peculiarities which means a bit of manual fettling is needed.

From my experience it seems a lot of the mainstream folk have ...

2025-03-31T13:35:06Z

In reply to nevent1q…h5qw
_________________________

From my experience it seems a lot of the mainstream folk have moved on to BlueSky, so it tends to be more tech-knowledgeable folk here.

If tech-knowledgeable fits the community, there's a number of fediverse compatible implementations listed at https://www.reddit.com/r/fediverse/comments/1i9bgs8/list_of_fediverse_software/ that focus on different types of media.

Is there a guide others can follow that shows how to keep issues, ...

2025-03-27T22:06:01Z

In reply to nevent1q…t9p7
_________________________

Is there a guide others can follow that shows how to keep issues, PRs, etc.?

#define OK 200 what could possibly go wrong? :)

2025-03-27T09:02:46Z

In reply to nevent1q…ap9r
_________________________

#define OK 200

what could possibly go wrong? :)

Nothing annoys me more that pop-up, modal, adverts in a service ...

2025-03-27T07:29:12Z

Nothing annoys me more that pop-up, modal, adverts in a service that is already being paid for.

#Google #Workspace #NoToGemini

The next step in the great, long-term, #Android bait & switch ...

2025-03-26T19:42:56Z

The next step in the great, long-term, #Android bait & switch :( https://www.androidauthority.com/google-android-development-aosp-3538503/

Completely agree, but I’m guessing Apple needs a way to stop ...

2024-12-31T18:57:17Z

In reply to nevent1q…f58g
_________________________

Completely agree, but I’m guessing Apple needs a way to stop people signing up, binge watching in a month, then cancelling.

Silo is worth a try.

2024-12-31T18:54:22Z

In reply to nevent1q…gjdq
_________________________

Silo is worth a try.

Looks like the Khadas Vim3 board is on sale in a popular global ...

2024-11-21T19:48:00Z

Looks like the Khadas Vim3 board is on sale in a popular global marketplace .

That board seems to run #Android well and is on the #AOSP reference board list.

Personally I’d pick the “Pro” version. 😉

https://source.android.com/docs/setup/create/devices

I never thought we'd reach a point where cars lose software ...

2024-09-09T07:51:16Z

In reply to nevent1q…hmhn
_________________________

I never thought we'd reach a point where cars lose software features as they get older :(

Does anyone at #Volvo really believe that, after 4 years, folk ...

2024-09-09T07:34:14Z

Does anyone at #Volvo really believe that, after 4 years, folk will pay a subscription to access services you can get for free via #AndroidAuto?

I can’t see locking this functionality behind a paywall, after 4 years, playing out well with consumers.

… and so the sun sets on my #UK holiday, and it’s back to ...

2024-09-08T15:50:12Z

… and so the sun sets on my #UK holiday, and it’s back to work tomorrow.

If you’re not had a holiday in a while, then just do it. If nothing else it helps to identify things that your on the hot path for, which you should be looking to have a “back up” option for.