Andrew Zonenberg

Security and open source at the hardware/software interface. Embedded sec @ IOActive. Lead dev of ngscopeclient/libscopehal. GHz probe designer. Open source networking hardware. "So others may live"

Toots searchable on tootfinder.

Public Key

npub1cddglts94qutscms0qpmk87lel9m8xku7q0wr20u2th5fxvvunqqxz9vpd

NIP-05 Address

azonenberg@ioc-exchange.mostr.pub

Profile Code

nprofile1qqsvxk504cz6sw9cvdc8sqamrl0uljanntw0q8hp48799m6ynxxwfsqpz3mhxue69uhhyetvv9ujuerpd46hxtnfduqs6amnwvaz7tmwdaejumr0ds75w78g

Publishing to

relay.damus.io

nos.lol

Author Public Key

npub1cddglts94qutscms0qpmk87lel9m8xku7q0wr20u2th5fxvvunqqxz9vpd

Show more details

Last Notes

2026-05-04 18:17:53 UTC

- reply

@nprofile…02n5 something something taylor swift private jet

2026-05-03 18:51:26 UTC

Whenever me and my wife are driving somewhere together and are backing out of a driveway, making a turn at a sketchy intersection, etc whoever is in the passenger seat always acts as a second set of eyes calling out traffic or "clear right" etc as appropriate.
From what I've seen this is common practice in military and fire department circles but pretty rare in the civilian world for some reason. Not sure why, better situational awareness definitely makes things safer.

2026-05-02 15:17:16 UTC

- reply

@nprofile…a65n it claimed submersible and i certainly got it pretty wet over the years. Pretty sure it's been submerged at least once.

2026-05-02 14:25:21 UTC

- reply

@nprofile…a65n air quality + IP rated might be harder since you'll need the sensor exposed

2026-05-02 14:03:39 UTC

- reply

@nprofile…a65n doesn't exist afaik

2026-05-02 08:57:47 UTC

I have a pair of end-of-life phones (CAT S61 and S62) with FLIR Lepton thermal cores in them.
Anybody know if there is a good way to disassemble them to salvage the thermal modules without destroying them? I don't care about the rest of the phone, it's gonna end up in ewaste.

2026-05-01 16:41:37 UTC

- reply

@nprofile…3tsk lol moss is native to this area the grass is probably what doesn't belong

2026-05-01 14:48:34 UTC

Having a bit of fun with my 100G LAN, new GPU, and a pair of ThunderScopes.
Here's not one but *two* 1 Gsps live waveform streams being FFT'd simultaneously in real time.
https://files.ioc.exchange/iocexchange/media_attachments/files/116/499/861/404/440/171/original/373d7d2f9bf5bc82.png
https://files.ioc.exchange/iocexchange/media_attachments/files/116/499/862/680/763/256/original/483932b71acf3a52.png

2026-05-01 03:44:30 UTC

Weird things that happen when you have a 100GbE pipe to your desk. 8 Gbps of sustained network traffic and the network monitor is like "yeah you're not using much bandwidth"
Also I think there's a 32-bit overflow or something in xfce4-netload-plugin because the rate shows 0.00 Mbps when I get above some threshold (not sure what it is exactly but it's in the 15-40 Gbps range)
https://files.ioc.exchange/iocexchange/media_attachments/files/116/497/247/334/128/502/original/8f0c619c80aec6bf.png

2026-05-01 01:49:36 UTC

- reply

@nprofile…r4z3 @nprofile…r9vg I've described my setup as "qubes on private cloud".
Same idea of isolated vms per security domain except i run them on a shared server and ssh+vnc to them so i can have one mail client, messaging app, etc shared among all of my endpoints

2026-04-30 16:58:54 UTC

- reply

@nprofile…r9vg I run the vast majority of my stuff that opens a socket to the internet in VMs that I can reboot much more casually.
But also I'm mostly running single-user systems. very few kernel bugs are RCEs, those are major headlines and the kind of thing that *would* get me to do an out of cycle reboot. But something that lets you privesc to kernel while running as a normal user really doesn't matter to me.
If you pop my workstation you a) can get all of my files and data, root doesn't buy you much else and b) it's a dev box I almost always have a root shell open or am going to su/sudo in the next few minutes to do something you can keylog.
The goal is to prevent an attacker from getting code exec as my local user in the first place not preventing root once they do that.

2026-04-30 16:08:04 UTC

- reply

Do normal arch users just reboot constantly? I'm trying to imagine the UX of installing a random package and having vulkan suddenly stop working until you reboot because your kernel driver and userspace are out of sync or something like that

2026-04-29 03:50:19 UTC

- reply

@nprofile…rues Intel XL710 -> Mellanox ConnectX 6. Haven't checked specs but it's likely faster, I think the old NIC was a gen3?

2026-04-29 03:46:30 UTC

Well this was not expected.
The 100G optics I ordered for my desk came in, and swapping the pipe to my desk from 40 to 100G gave a huge improvement in Ceph performance.
But I don't understand *why*.
Baseline config from earlier in the year: client on 40G, cluster nodes on dual 10G, 1558 MB/s on linear reads
Moving cluster nodes to 40G, client on 40G: 1864 MB/s
Moving client to 100G, cluster nodes on 40G: 3787 MB/s.
The confusing thing is, even 3787 MB/s is only about 30 Gbps, so after protocol overhead I would expect it to fit comfortably in 40G. Why can I get this performance with the client on 100G, but not on 40?
https://files.ioc.exchange/iocexchange/media_attachments/files/116/485/423/725/240/394/original/9cc10ec6167dd1c0.png

2026-04-27 06:33:02 UTC

Anybody familiar enough with Supermicro main board architecture to know what they're using the little Lattice CPLD for? Just IO expansion for the BMC or what?

2026-04-27 03:32:13 UTC

- reply

Quick summary from preliminary doc reading having not tried any of them yet:
* VMware does not appear to support virtualizing macos apple silicon guests
* Parallels appears to be subscriptionware but will probably do what I want if I buy the pro edition (seems the lower tier doesn't have the CLI)
* UTM is free and seems to support MacOS guests, so I'm gonna try to install it and see what happens

2026-04-27 02:39:22 UTC

- reply

It looks like the players so far are vmware fusion, UTM, and parallels

2026-04-27 02:38:37 UTC

What's the go to for virtualization on apple silicon macs?
Requirements:
* Free or one time purchase, not subscriptionware
* Can run macos guests with full metal support
Stretch goals:
* Can run arm64 Linux guests with vulkan support

2026-04-26 11:50:12 UTC

- reply

@nprofile…4424 let me put it this way if you are the kind of shop that runs cadence virtuoso on linux, you can afford a RHEL license :p

2026-04-26 11:48:09 UTC

- reply

@nprofile…4424 yes RHEL is what the big eda folks run for linux, fedora is close enough we expect it to work on RHEL despite not officially testing

2026-04-26 11:44:45 UTC

- reply

@nprofile…4424 We do have some, yes. Not sure how many.
Arch has caused me a lot more pain than Fedora to date, mostly due to random package updates causing breakage but also the recent llvmpipe saga causing the GitHub release status to show as failing for the past few months (I might just disable that test suite on GH for now).

2026-04-26 09:35:20 UTC

- reply

So now we have builders for
* arch + nvidia
* debian 13 + nvidia
* debian 12 + nvidia
* ubuntu 26.04 + llvmpipe
* ubuntu 24.04 + nvidia
* win11 + nvidia
Fedora is next on the list as soon as I fix an issue around updating the msys2 runtime on the Windows build...

2026-04-26 09:31:47 UTC

- reply

I ran a few quick calculations on power consumption: measuring at the UPS, the baseline consumption of the lab a minute or so before kicking off a CI job was 3828W.
During testing it hovered around 3900-3950, with a peak of 4007.
Using worst-case numbers assuming (4007-3828) = 179W extra power consumption during the build compared to idle, sustained over 17 minutes, a build would use about 0.05 kWh of power, which as of my last bill would cost me about one cent. Since the actual build is not running at that peak power the entire time, the actual energy requirement is substantially less.

2026-04-26 09:28:56 UTC

OK, the Arch runner is now set up and working after my rather annoying tangle with llvmpipe.
The entire six-job build ran for 17 minutes. I could probably optimize it further: right now the Windows and Ubuntu 24.04 builds are among the slower ones and they share the same GPU forcing them to run sequentially. If I shuffled around which VM ran which job, I could probably fine tune a bit.
Max concurrent jobs was four, three using nvidia cards plus the Ubuntu 26.04 using llvmpipe. I was watching YouTube videos in a SSH+VNC session to another VM on the server the whole time and it didn't stutter or slow down noticeably, suggesting that I wasn't overloading the server.
I think my earlier problems with performance of other VMs during builds were due to the default xcp-ng scheduler granularity being "CPU" not "core"? After changing that things seemed to be a lot better. Was it maybe allocating the entire 32-core CPU to a single VM at a time or something in that mode??

2026-04-26 08:55:14 UTC

- reply

@nprofile…a65n well i dont have easy access to anything else mesa supported on this vm (or any arch instance i have access to) and nvidia doesnt use mesa.

2026-04-26 08:50:56 UTC

- reply

@nprofile…a65n I was able to initialize vulkan and run a couple of shaders without any problems; 13 of my 17 unit tests crashed with weird pointer or memory corruption looking faults in code that seemed to be llvmpipe's JIT binaries from my SPIR-V.
When I put an nvidia card in the same VM, it runs without issue

2026-04-26 08:49:19 UTC

- reply

@nprofile…a65n The crash I was getting was segfaults deep inside LLVM-generated code with no symbols, and a thread named "llvmpipe".
I don't have a full vulkaninfo dump from the VM at that snapshot handy but here's a subset of the info https://dashboard.ngscopeclient.org/tests/920

2026-04-26 08:45:20 UTC

- reply

@nprofile…a65n well i dont even have a display server *installed*, this is a headless VM, so it shouldn't be confused by that

2026-04-26 08:35:20 UTC

- reply

@nprofile…a65n there is no x or wayland server running so i dont think there is any mesa? or is mesa involved in the vulkan stack still even when nothing is being drawn?

2026-04-26 08:34:42 UTC

- reply

@nprofile…a65n I do not have any AMD cards in the VM server at the moment only nvidia.
So my options are currently llvmpipe or a GTX 1630.
Down the road when I shuffle some stuff around and put a 100G NIC in the box, I am considering trying to squeeze an AMD in to enable testing on a larger cross section of platforms

2026-04-26 08:13:17 UTC

Is anybody using llvmpipe on Arch Linux with Vulkan? Have you tested recently?
I'm seeing all kinds of horrible segfaults of trivially simple shaders in ngscopeclient + llvmpipe with Arch on both my own infrastructure and GitHub Actions.
When I put a nvidia card on my Arch VM and use their Vulkan driver instead, everything works fine.

2026-04-23 03:17:56 UTC

- reply

@nprofile…7l00 HIPAA doesn't prohibit collection of information, only how it can be disclosed once it's collected.
A lot of ambulances have cameras in the back too.
There are plenty of legitimate uses besides the "safety/security" justification that's often used; it lets you replay an incident and debrief it to find things you might have missed in the heat of the moment that could have been done better. I know of one service (Parker County) that uses body cam + ambulance cam footage for internal debriefs on all of their cardiac arrest / CPR calls. This helps improve patient outcomes and seems like an entirely reasonable use for the data; it's only being shown to people who were there anyway and is just serving to jog their memory.
If the patient consents to a broader release of the footage, of course, then they can do whatever the patient allows them to.
Which is how you get really cool videos like this https://www.youtube.com/watch?v=p_Fp2hhUPK8 that run through the entire call end to end. They actually have a couple of different ones on this particular call with various analysis and narration by the medics who ran the call, their medical director, and some other folks at the agency.

2026-04-21 15:48:40 UTC

- reply

@nprofile…4424 I have a TBT to 10G SFP+ dongle on my work laptop and keep meaning to put an intel 10G controller asic in a custom framework expansion bay board so i can have SFP+ to the other laptop natively with no dongles

2026-04-21 15:38:14 UTC

- reply

@nprofile…4424 10gbaseT is way worse I don't use it because it's trash

2026-04-21 15:37:59 UTC

- reply

@nprofile…4424 lol oh when i have my switch prototype on a metered lab supply i can very clearly see the few hundred mW from a 1000baseT link toggling

2026-04-21 10:10:24 UTC

- reply

@nprofile…4424 to give you a better idea...
The barely perceptible jump around 21:00 is, I think, me turning on the switch,
Compare this to the much sharper and more well defined jump around 01:00 where I was playing a game on my desktop causing increased CPU/GPU load.
Assuming worst case 75W 24/7 that's 1.8 kWh a day, or about $11 a month at my current power prices. But that's a worst-case number and I expect the actual number will be quite a bit less.
Put a different way, this time of year on a sunny day *one* of my solar panels puts out enough power during daylight hours to run this switch for the entire day, including overnight.
https://files.ioc.exchange/iocexchange/media_attachments/files/116/442/129/582/012/248/original/059d392f6d57ac1c.png

2026-04-21 09:28:19 UTC

- reply

@nprofile…4424 (that should go down now that the sun is shining more)

2026-04-21 09:27:04 UTC

- reply

@nprofile…4424 I mean I have a $800ish power bill it's hard to see a load this small

2026-04-21 09:19:06 UTC

- reply

@nprofile…4424 75w is datasheet max. ups is metered but there's 4 kW of load tens of watts are lost in the noise

2026-04-21 07:37:31 UTC

- reply

I wish "show int status" output on the FS switch showed the vlan number that would be helpful to have at a glance.
https://files.ioc.exchange/iocexchange/media_attachments/files/116/441/539/407/015/835/original/6bca3c967441b9d7.png

2026-04-21 07:30:31 UTC

- reply

I might have to do some tuning on the Ceph cluster to push performance up a bit more, though.
Right now with all 3 cluster nodes and the client on 40G, I get 991 MB/s (7.92 Gbps) of writes, 1864 MB/s (14.9 Gbps) of sequential reads, and 1841 MB/s (14.7 Gbps) of random reads.
This is certainly "usably fast" but not saturating the 40G like I wanted/

2026-04-21 07:27:32 UTC

- reply

There are of course a lot more ports wired up and just not actively lit, I have OM4 all over the house going to wall ports around the lab and office that I can hook things to when experimenting with an FPGA board or something.
But this is a nice near-term upgrade.

2026-04-21 07:25:05 UTC

- reply

After the Ceph node migration I now have only 11 active 10G SFP interfaces on the Nexus, one of which will be going away when I swap the VM server over to 40G.
Of the remaining 10, only 6 are real 10G loads and the other 4 are 1000baseX links to the 1G edge switches.
But once I deploy my own switching for the 1G edge, they'll have real 10/25G uplinks

2026-04-21 07:21:28 UTC

- reply

All wired up, now using 5x 40G + 1x 100G ports for the moment. Office workstation will get bumped from 40 to 100 at some point so then I'll have two free 40G ports, one of which will eventually get hooked to the VM server.
Plus three free 40G ports on the 10G switch.
Also I picked a hostname, it's argon2
https://files.ioc.exchange/iocexchange/media_attachments/files/116/441/480/998/810/783/original/5a71fe4ac4c7006d.jpg

2026-04-21 05:52:40 UTC

- reply

On this switch, trunks default to *no* vlans allowed. You have to do "switchport trunk allowed vlan all" before it will pass any traffic.

2026-04-21 03:24:41 UTC

Getting ready to deploy the new switch and realized a major problem: it doesn't have a hostname yet.
My general theme for switch hostnames is hash algorithms although I've expanded a bit due to exhausting the common ones so I'm using MACs etc as namesakes too.
My current fleet is keccak, blake, skein, poly1305, and ethash. All of the SHA/MD algorithms are previously used.
https://files.ioc.exchange/iocexchange/media_attachments/files/116/440/553/417/582/155/original/783c4571149902dc.jpg

2026-04-20 06:07:58 UTC

- reply

Spider Georg (who lives in a drain not a cave and definitely does not eat 10,000 other spiders a day) came out of the drain for a bit and I got a pic.
Pretty sure he's a hobo spider or giant house spider but not 100%. He can't get out of the tub because the walls are too slippery (I've seen him try) so whatever he is, he's stuck in there lol.
https://files.ioc.exchange/iocexchange/media_attachments/files/116/435/521/250/535/642/original/7fc70a691494e8f0.jpg

2026-04-20 03:11:09 UTC

740.3 kWh month to date. more than we've got during all of March. Solar power season is definitely starting to hit, we're on track to be generating something around 1.2 MWh this month.
Of course this is nowhere near enough to equal my projected consumption of ~4.8 MWh but it'll certainly put a dent in my power bill.

2026-04-20 00:56:13 UTC

- reply

His name is Georg.
Because he is a statistical outlier and should not be counted

2026-04-19 19:46:34 UTC

Our downstairs bathroom has become a de facto half bath since the shower head developed a leak at the elbow that I can't repair without ripping out the water heater and a wall in the laundry room.
This little guy moved into the tub and seems to be doing pretty well for himself. Sometimes he climbs out of the drain and scuttles around the tub but most of the time he seems to like dangling from the drain by two legs.
https://files.ioc.exchange/iocexchange/media_attachments/files/116/433/079/070/286/386/original/f8709759d089d943.jpg

2026-04-19 12:02:42 UTC

- reply

@nprofile…u57d I'm pretty sure the last unsolicited call I got from work was when the low-oxygen alarm in the mechanical room went off at like 5:30am when one lab tech was the only guy in the building.
Luckily it turned out to be the sensor hitting end-of-life and bricking itself rather than going out of calibration (because apparently this sensor had slipped through the cracks and not got the regular maintenance it was supposed to), not an actual N2/Ar leak.

2026-04-19 11:46:14 UTC

- reply

Ok all reassembled and patched and took care of some other housekeeping plus reopened most of the wip stuff i wanted to keep active.
Now it's almost 5am and I'm about where I wanted to be at 2200 last night (pcie all shuffled, 100g nic installed in last cluster node, desktops functional)

2026-04-19 10:12:24 UTC

- reply

And the second one is now POSTing after the BIOS reflash too.
Just gotta put the GPU and rest of the RAM back in that I took out while troubleshooting.
Seriously though, *how*???

2026-04-19 04:59:36 UTC

Poking around a bit more at the PCIe configuration on my storage cluster nodes and between the UEFI config and the manual I think I fully understand the topology now.
(do consumer mainboards ever publish this kind of info? I'm gonna guess no... but I havent bought one in ~10 years)
The CPU itself has 48 PCIe lanes, which was the most available when I deployed these (icelake-SP bumped this up to 64, and sapphire rapids to 80).
They're divided into three x16 root complexes.
Group 1 goes into a mux (Br3) which either connects to slot 1 as x16, or bifurcates to x4 to each of the four M.2 ports.
Group 2 goes into a Broadcom PCIe switch which splits off to four slots 5:2. These slots are paired, and each pair (2/3 and 4/5) can be either dual x8s or a single x16, but all four share the single x16 to the root.
Group 3 goes to a mux (Br2) which either connects to slot 7 as x16, or slots 6/7 as dual x8. It can also bifurcate to quad x4+x4+x4+x4 in slot 7 or dual x4+x4 in each of 6/7.
https://files.ioc.exchange/iocexchange/media_attachments/files/116/429/518/204/191/700/original/1a697eb27094959d.png

2026-04-18 20:04:20 UTC

- reply

@nprofile…2u6u No recording, only slides

2026-04-18 01:52:35 UTC

- reply

@nprofile…thd8 as usual the long answer is "there's no closer form solution because it's stackup and geometry dependent so just throw it in a field solver and when the plots with/without the in plane metal look close enough that's your answer"

2026-04-18 01:50:38 UTC

- reply

@nprofile…thd8 3x the distance to the ref plane is a reasonable first order estimate but e.g. half oz vs 2oz copper will have a big difference in edgewise coupling strength

2026-04-18 01:49:19 UTC

- reply

@nprofile…thd8 there's no hard cutoffs since fields never decay fully to zero at any distance.
The practical answer is, the point at which the coupling in-plane becomes negligible compared to the coupling to the reference plane. This is dependent on copper thickness, soldermask presence and Dk compared to substrate Dk, etc.
And exactly how "negligible" you want to let the coupling get before you can ignore it

2026-04-17 22:43:36 UTC

- reply

This is very much a WIP, we don't have a full break of the target (it's been kind of a back-burnered project around other stuff) but I thought the obfuscation was worth talking about just because there's so little detail on this sort of tradecraft out in the public sphere

2026-04-17 22:39:50 UTC

Slides from my HARRIS 2026 talk, "Anti-RE Countermeasures in a Real Secure Element", are up at http://harris-frab.mpi-sp.org/media/harris26/question_uploads/harris2026-zonenberg_sM1gyme.pdf

2026-04-17 08:12:07 UTC

Has anyone done a test implementation of https://www.ietf.org/archive/id/draft-meow-mrrp-00.html yet? Is there any open example code?

2026-04-16 08:17:56 UTC

- reply

@nprofile…5a85 My solar system is grid tied and no use without utility power, although if I get a battery to go with it that will change and I'll be able to run off grid in a pinch supplementing with generator.
My current plan is... for brief blips in power the UPS handles things for the computers, nothing else cares.
For longer duration (30 minutes to maybe one day) outages I fire up the generator to keep everything running so I can work, not lose open documents, and remain fully productive. The only things that degrade in this state are the heat pump, water heater, and oven - all of which use too much power to run on generator.
If I anticipate the outage being multiple days in length (i.e. it's the result of a large scale disaster not a random storm or tree on a power line), we're no longer in "business as usual" mode.
In that case priority #1 is to shut down all of the tech as quickly as possible to keep the batteries charged for later.
Then go dark but set up the generator. A few times a day, fire it up to cool down the fridge and run the microwave or toaster oven or something to eat as much perishable food as we can (charging phones etc at the same time), then shut it off and keep the fridge closed to stretch our limited fuel supply as long as possible.
When we run out of generator fuel or fresh food, switch to shelf stable stuff.

2026-04-16 06:47:25 UTC

- reply

@nprofile…z3nt @nprofile…a638 Sure, by all means be prepared to escape and/or resist if a band of violent looters shows up.
But you also have to consider the more practical issues - some people might have medications that need refrigeration. Perishable food is going to spoil unless you can provide backup power to refrigerators and freezers. But if you can't preserve that food, you're not going to be able to last nearly as long on shelf stable pantry goods. How much water do you have? Is it drinkable?
I'm downhill of a water tower so even if treatment plants and pumps fail, we would probably still have water pressure at least for a while, but if pipes are damaged it might have to be filtered or boiled before it's safe to drink. Who has stockpiles of bottled water, and how much?
Who's injured? How bad? Who has first aid or medical training, and supplies? Can people be treated for minor cuts and scrapes and sent home unsupervised? Can someone with a broken limb be splinted and left with a friend until you can get them to proper care, potentially days out? Is there something really critical that demands immediate higher care, and is there anyone nearby capable of providing it? Can you get them there on foot?
Sure, tourniquets and quik-clot are nice to have. But can you manage someone with a broken arm for a week until a boat shows up to take them over to the one undamaged hospital in the area?

2026-04-16 06:39:26 UTC

- reply

@nprofile…z3nt @nprofile…a638 But the part they miss is that 99.9% of the time, going solo isn't the way to do it.
The most likely scenario is that a town or neighborhood will have most or all paths in and out cut off, and some or all utilities out of service for an extended period of time.
So what you need to do is address any imminent life threats (put out fires, evacuate unstable structures, rescue people trapped under debris) to the extent possible, then get together with neighbors (ideally not meeting them for the first time) and make a plan to deal with whatever is happening right where you are, because you're all you've got.
The three fire stations in my city of 13000 are not sending anyone to my tiny street when the big one hits regardless of how bad the damage is or how many injuries there are. There's too many of us and not enough of them.

2026-04-16 00:37:23 UTC

Predictable network interface names strike again. Removed a 10G NIC from a box and the 100G NIC that was enp27xxxx is now enp26xxx because PCIe enumeration changed

2026-04-13 00:24:22 UTC

Apparently the creationists have pivoted from preaching to building garden products
https://files.ioc.exchange/iocexchange/media_attachments/files/116/394/546/922/313/347/original/af7198fc4c5cd394.jpg

2026-04-12 20:21:42 UTC

Anybody know of a vendor that has dual socket Ice Lake server motherboards available? I can find uniprocessor ones without much trouble, but dual socket ones are in much shorter supply.
It's not *that* old a platform, there should still be some floating around...

2026-04-12 05:03:30 UTC

Firestopping on cable tray penetrations between my lab and house (cc: @nprofile…vnlz )
https://files.ioc.exchange/iocexchange/media_attachments/files/116/389/978/735/148/892/original/f863a5833703c49f.jpg
https://files.ioc.exchange/iocexchange/media_attachments/files/116/389/979/177/788/957/original/bff2553b16e50a50.jpg
https://files.ioc.exchange/iocexchange/media_attachments/files/116/389/979/632/629/598/original/8ad0dae4ed417e90.jpg
https://files.ioc.exchange/iocexchange/media_attachments/files/116/389/980/031/748/442/original/6f241b72195eb11e.jpg

2026-04-11 17:46:19 UTC

- reply

@nprofile…tw96 you have to figure out which disks are os images vs delta images for snapshots or memory images for suspend. They're all just uuids.
I don't know how to remap the delta images and expect to lose all of my snapshots.
And I can't find the image for the chat vm home partition at all. Unsure if corrupted somehow or what but i mounted everything I could and didn't find it

2026-04-11 16:47:38 UTC

21 VMs recovered, everything important except the messaging VM. If I can't recover the /home disk I'll lose some chat history but that's not the end of the world. I'm not sure what's going on there I've gone through what I think is all of the disks and none of them seem to be it.
And then the CI runner VMs seem to have issues related to snapshots that I'm still chasing, but those aren't going to be that hard to recreate if I have to.
Haven't touched any of the Windows boxen yet.

2026-04-09 21:43:24 UTC

References to specific paragraphs and clauses of IEEE 802.3 is a perfectly normal thing to find in a GLSL shader, right?
Just normal shader developers doing normal shader things.
https://github.com/ngscopeclient/scopehal/blob/master/scopeprotocols/shaders/Ethernet100BaseT1_Decoder.glsl?ts=4#L100

2026-04-08 23:30:38 UTC

It's... Bigger than i thought it would be.
About time I had one of these. Quite the upgrade from my old clicky thing that doesn't go low enough for small screws on PCBs etc
https://files.ioc.exchange/iocexchange/media_attachments/files/116/371/683/033/886/488/original/43d08f911e8e7f39.jpg
https://files.ioc.exchange/iocexchange/media_attachments/files/116/371/683/377/578/556/original/3355666c95cd05ee.jpg

2026-04-07 23:15:30 UTC

- reply

@nprofile…fz8u yep, the good ones are wraparound and dont leave any exposed adhesive.
Heatshrink collars only make sense if you are putting them over bare cat5 before terminating it

2026-04-07 23:12:05 UTC

- reply

@nprofile…fz8u factory serial numbers (on premade cables) would do that
but these days i just use a label maker and give each cable a clear marking of the source and dest on a heatshrink collar or wraparound sticker

2026-04-07 19:25:12 UTC

- reply

@nprofile…t2th Generally speaking, most of the existing blocks prefer GPU implementations as we are aggressively trying to go 100% GPU, and GPU memory bandwidth is so high that you are often better off copying data to the GPU and doing simple transformations there and only moving back to CPU at the last second if you need to do that at all.
Simple intermediate blocks like basic math are very important to do GPU side even if there isn't much compute (like pairwise adding two waveforms) to avoid GPU-CPU-GPU paths incurring large transfer delays.
Where it gets *fun* (a currently unsolved problem that is going to require a lot more rearchitecture) is going multi GPU and scheduling workloads when your data may be resident on different cards, moving data between them is expensive, and you need to schedule tasks based on predictions of how long it will take for a shader to run vs how long a copy will take to know whether you are better off submitting work to the GPU the data is resident on (to avoid a transfer) or another GPU (incurring a transfer but allowing both to be busy with shaders simultaneously).

2026-04-07 16:34:40 UTC

- reply

I tweaked a few things and added some barriers and on one execution (nondeterministically, of course) I got *both* buffers to have the same value, which matches the hash of what should have been in vectorTempBuf1 (dinFwd).

2026-04-07 16:24:41 UTC

Finally some progress. But it's baffling.
The very first step of the de-embed filter is to execute FFTs of dinFwd to vectorTempBuf3 and dinRev to vectorTempBuf1.
It appears that somehow the FFTs trade places in the failure scenario, with the frequency domain version of dinFwd being written to vectorTempBuf1 and dinRev going to vectorTempBuf3.
I don't even know where to start debugging this, it smells like something going wrong in the depths of vkFFT with some state that should be per-plan being global or something?
https://files.ioc.exchange/iocexchange/media_attachments/files/116/364/340/723/816/716/original/2ded82b306ec12f4.png

2026-04-07 03:06:25 UTC

- reply

@nprofile…fz8u i read that as "feral farmer" and that fits you just as well

2026-04-06 04:59:02 UTC

- reply

Noise waveforms seen by ZS1500.
They get stronger when the probe is sitting flat on the steel lab bench and weaker when it's lifted up on top of something.
https://files.ioc.exchange/iocexchange/media_attachments/files/116/355/985/786/971/579/original/77089ec95f74108b.png
https://files.ioc.exchange/iocexchange/media_attachments/files/116/355/985/774/881/865/original/f48316d2617db98a.png
https://files.ioc.exchange/iocexchange/media_attachments/files/116/355/985/785/144/275/original/e097be7bf11e0e8c.png

2026-04-06 04:37:13 UTC

- reply

like seriously how am I getting 40 mV p-p of noise *through the directional coupler with 10 dB coupling loss* coming out of my baseT1 PHY when the board isnt even powered up

2026-04-06 04:35:30 UTC

- reply

Noise waveform with the media converters powered off but the RJ45 connected
https://files.ioc.exchange/iocexchange/media_attachments/files/116/355/898/559/216/646/original/53a185eadad854bb.png

2026-04-06 04:32:36 UTC

- reply

Unplugged the RJ45 from the right hand media converter (the long haul one going into the in-wall cable plant) and the noise got a lot smaller.
Which is baffling because that RJ45 has no DC connection to anything in the DUT, it's just going into a magjack which drives the baseT PHY.

2026-04-06 04:28:57 UTC

- reply

Unplugged channels 3 and 4 of the TS from the fixture and I no longer see the noise bursts on those channels.
So it's not the scope doing it. I'm measuring real noise from somewhere.

2026-04-06 04:27:28 UTC

- reply

The overall noise waveform appears to be ~2.8 ms off, ~2.9 ms of stronger noise, and ~2.3 ms of the strongest noise then repeating.
The noise bursts consist of 38 kHz repetition pulses with ~356 kHz narrower pulses between them. The highest frequency noise is around 4 MHz.

2026-04-06 04:25:24 UTC

- reply

The plot thickens.
Unplugged power from both media converters but left everything cabled up as-is.
I see the same noise spikes.
Channels 1/2 (A direction) and 3/4 (B direction) are seeing approximately the same noise waveform, but with reversed polarity.
https://files.ioc.exchange/iocexchange/media_attachments/files/116/355/857/172/980/623/original/420be0f185d90d37.png
https://files.ioc.exchange/iocexchange/media_attachments/files/116/355/857/194/793/907/original/933816763a28aebe.png

2026-04-06 04:10:11 UTC

- reply

Shortened the green leg on the other side of the fixture doesnt seem to have helped much.
One thing I did notice that doesnt show up well in still screenshots is the dynamics. It seems like the signal is jumping between multiple amplitudes which makes no sense.'
Not like PAM, like the P-P amplitude is not constant.
https://files.ioc.exchange/iocexchange/media_attachments/files/116/355/796/925/423/711/original/3a3db6465f29acd1.png
https://files.ioc.exchange/iocexchange/media_attachments/files/116/355/799/007/049/020/original/ce224fe2652c938a.png

2026-04-06 04:05:47 UTC

- reply

Next, let's try shortening the UTP runs to see if they're coupling noise from something nearby into the DUT.
Shortening the orange leg seems to have improved things a bit?
https://files.ioc.exchange/iocexchange/media_attachments/files/116/355/781/495/645/395/original/79d8b472c0cbc843.png

2026-04-06 04:01:02 UTC

- reply

@nprofile…rues I wish I was exporting that much, lol.
I've consumed 786 kWh month to date while exporting 10.3 kWh and producing 169, for a net import of 618.
This is 41x Sil-Fab 440 QD on IQ8AC, no battery. Lots of tall trees plus cloudy weather earlier in the year but weather has been mostly decent this month except the 1st which was kinda cloudy.

2026-04-06 01:59:19 UTC

It's April 5th and I've generated 168 kWh of solar power this month.
Compare this to the 146 kWh I generated all of January, lol. To be fair the array was offline awaiting inspection between the 14th and 25th so I was only producing for 19 of 31 days, but still... extrapolating from the average, I would expect to produce around 238 kWh next January assuming similar weather, and I should reach 238 of generation for April some time Tuesday afternoon.
Using the extrapolated numbers for January, we get the following numbers for daily production as well as long-term average power output (averaged over 24 hours).
January - 238 kWh/31 days, 7.6 kWh/day, 316W average
February - 393 kWh/28 days, 14.0 kWh/day, 585W average
March - 731 kWh/31 days, 23.5 kWh/day, 979W average
April (month to date) - 168 kWh/5 days, 33.6 kWh/day, 1.4 kW average

2026-04-06 00:45:26 UTC

- reply

You know, I could see there being one in a military somewhere. A field hospital trauma surgeon who also does buzz cuts when there's no patients.

2026-04-06 00:44:12 UTC

- reply

There has to be one, somewhere, even if they just do it for the humor value

2026-04-05 04:14:06 UTC

🎶 Cut diffpairs into pieces
This is my board rework

2026-04-03 00:17:35 UTC

- reply

@nprofile…gftk I've diagnosed similar reflow defects in the past by elimination (drive the pin high, see the trace isn't high, etc). In ~every case it was due to me terminating the reflow too soon on a high thermal mass board.
The fix was just to squirt some flux under the chip with a syringe from all sides and do a second, longer reflow cycle and in every case it was fine afterwards.
Doesn't happen often now that I have my setup pretty dialed in but on my old single element no-convection toaster oven I remember doing it a handful of times.

2026-04-03 00:14:04 UTC

- reply

@nprofile…gftk Fun, lol.
Anyway it should be pretty trivial to rework (squirt some flux under it then hit it with hot air). Somebody on linkedin suggested it when I was posting around soliciting fun defects to put on the rework challenge and I figured sure, why not give it a try.
The intent here is to simulate a solder paste printing issue or some kind of reflow problem. It would definitely be hellish to debug but luckily this is a rework challenge not a debug one, so we tell you which balls are bad and you just have to fix it

2026-04-03 00:08:51 UTC

- reply

The simulated reflow defect is the bit I was most concerned about: I put no solder paste apertures on those balls in hopes of creating an open circuit failure, but I didn't know if I might have a good-enough contact between the ENIG and the SAC305 ball sans flux/paste to still make some level of contact.
I got exactly the result I wanted, it failed open.

2026-04-02 23:52:29 UTC

- reply

Banged up some quick grader firmware.
Confirmed challenges 3 and 13 (the only two I've attempted to bodge to date) are passing, all others failing as expected including the simulated reflow defect on balls A2 and A3.

2026-04-02 22:17:51 UTC

- reply

Post stream: the board is fully populated and challenges 3 and 13 have been (hopefully correctly) solved, but I won't know for sure until I get firmware written to electrically test the connections.
I've hooked up JTAG and the MCU is alive, still need to write firmware to actually grade the challenges so that will be the next step.

2026-04-02 17:19:45 UTC

Going live at noon Pacific (just under 2 hours from now): second stage of the ReworkCTF playthrough
Plan is to do the pre-assembly rework of the BGA site (challenge 13), populate the board, then get as far as I can working my way through the challenges.
https://www.youtube.com/watch?v=cuiM8zESF1w

2026-04-02 12:37:28 UTC

- reply

@nprofile…ds0l The only way it makes sense is if you already (before stealing it) have a buyer lined up who really wants it for themself, understands they'll never be able to show it off, and is willing to pay a high price.
I'm sure these exist but they're rare.

2026-04-01 21:05:34 UTC

Conn, sonar... Bratwurst in the water bearing 278
https://files.ioc.exchange/iocexchange/media_attachments/files/116/331/474/148/284/505/original/bdb59ba17790ac0d.jpg