Tootfinder

It's a tight fit, but finally got the PiKVM Switch stack sitting nice and flush in the basement.
#homelab #minirack

Is it questionable if you get more pleasure from updating your homelab than from a vacation trip?
#homelab #UpdateFriday

Okay, everything is buttoned up again, only almost stripped a rack hole out. However, time to leave things alone and see if it catches fire.
#homelab #minirack

Wooo, got the PiKVM on the OOB VLAN. That took way longer than it should've, but finally tracked down the small issue I had with the `systemd-network` files.
#pikvm #homelab #minirack…

When you wake up in the morning after a thunderstormy night and find the one electric fuse which cares for your homelab has triggered. The USV must have screamed as long as it could .. It's strange I didn't hear it.
After I switched power back on one of the proxmox nodes didn't come back. Dead.
Good morning everyone! It's monday!
#homelab
PS: It's…

The top MS-A2 server is very, very warm as it's ingesting heat from the RPis above along with the two switches. Grabbed a couple of fans, a 12V power supply, and a few fan screws. Give it an hour or two and see if I'll need to add something else or change the airflow.
#minirack #homelab

I hit a wall inspiration-wise for the last few months to hack on my #homelab. Not entirely sure why that happened, maybe research fatigue from planning for my Japan trip?
Anyway finally got back into it a little bit today as I moved forward on setting up my M4 MacBook Pro. It's been sitting partially configured for months now but spent some time applying my dotfiles config.

Migrated the homelab authenitcation from Authelia to Pocket ID. Fantastic user experience and *so* much easier to configure.
#homelab

Pocket ID

For the past month, I have been flitting about from one project to another as my ADHD sees fit. Today, that MO has to end if I'm ever going to complete this #homelab setup.
So, in order, I have to:
* Back up all my LXC's
* Install RAM in a Mac mini
* Reinstall ProxMox with #ZFS

So, went ahead and connected the HDMI cables to the PiKVM switch and at least got the video routed to the PiKVM v4. While these were pricey, the software and capabilities more than make up for it.
#homelab #pikvm

The last thing to install into the minirack: some nice remote access. Still need a few HDMI cables and I need to figure out a way to integrate the power button control into the PiKVM stuff, but baby steps.
#homelab

Alright, gonna do a quick test. I have the encode numbers for the `encoderXX` workers running Ubuntu 24.04 LTS and they're not great. Gonna build a node using CachyOS and see if that changes things.
#linux #homelab

Hrmm, it seems like there aren't many sources on how to put a PiKVM onto a proper VLAN. It seems like it's using `systemd-network` for networking, might have to put together an article on how to do it properly assuming I can remember correctly.
#pikvm #homelab

Started running some cables into the basement for the PiKVM. Got the network patch keystone installed in the patch panel, and ran the network connection along with the first two MS-A2 server HDMI cables. Unfortunately, I'm still waiting on the third HDMI cable, all of the USB-A to C cables, and a few replacement cables for the PiKVM KVM Switch. Things progress though...
#homelab

Moving some services into Europe to #hetzner
Step 1 completed… DNS moved and LetsEncrypt certificates for the #homelab configured
Now onto certificates for the new Hetzner server….

Alright, bumped the encoder VM from 8 to 12 cores, let's see if this helps out with the encode times. I have a sneaking suspicion it will not.
#homelab #ffmpeg

Pretty happy, first episode went from ~6GB to 250MB. Settings are still good, quality is outstanding, and the chapters, subtitles, and fonts all muxed in beautifully. Gonna let these encodes finish up on the cluster while I listen to some music and drink a few beers outside.
#homelab #encoding

Is there a self hosted Just In Time (JIT) PAM service?
#AskFedi #JIT #SelfHost #HomeLab #Cybersecurity

Went ahead and upgraded the Proxmox cluster from kernel version `6.8` to `6.14`. Seemed like a good idea, didn't have any issues with VM migrations so that was nice.
#proxmox #homelab

AV1 might take a ton of time to encode, but the results are awesome. I do need to take a look at the VMs though as I'm seeing a ~33% increase in encode times when compared to the previous nodes I was using. Either way, the minirack is still quiet and taking everything that's been thrown its way.
#homelab

First real test of the new cluster: encoding AV1 video. I created three Sisyphus encoding workers and decided to run some Blurays across all the servers. Wrote some Ansible to build the encoding workers because it's a pain in the ass, but right now I'm working through the first 3 videos in a queue of 12. Also, these are so much more efficient...
#homelab

The first MS-A2 node is up and it only took the entire day. After fixing a ton of networking issues across the homelab I've got all the VMs running on the new node. The old nodes have been decomissioned which kinda marks a milestone. Still need to get the other two new nodes into the cluster, but should be fairly straightforward and can wait until tomorrow.
#homelab

Multipathing: it makes fixing cabling issues so much easier. Swapped out the 2x2m fiber patches to the NAS with 2x5m ones because the old ones were just a smidge strained. Had absolutely no issues swapping them out one by one and no downtime.
#homelab #multipath

Old servers are prepped and ready for their new owner and new home.
#homelab

Eh, final home for the time being. Once the second nameserver finishes updating I can get the cables managed a bit better. Either way, officially done.
#homelab #minirack

Did a smidge of cable management because the fiber runs to the QNAP needed a bit more slack. Things I love: being able to disconnect the connections to the iSCSI storage and have absolutely no loss of connectivity.
#homelab

Got `acme.sh` installed on all the Proxmox nodes and managed to get the certs issued and installed to the right location without too much hassle. All of the UIs now no longer anger the Firefox gods. Also all of the servers are updated and nothing is on fire. Time to relax and debate on what to eat for supper.
#homelab

Hrmm, it seems that my Proxmox migrations are going over the 2.5GbE links. I'll have to fix that later so it uses the 10GbE links but it's definitely a low priority.
#homelab #proxmox

Networking is finished, and every port on the QNAP switch is in use along with every port on each MS-A2. Also tracked down an `interfaces` file typo. The next step is cleaning up the "server room" and getting this situated.
#homelab #minirack

Welp, gotta transfer these images over to the NAS. However, after that it's time to figure out networking.
#homelab #minirack

So, might've gotten a few things in the mail today...MUHAHAHA. The Minisforum MS-A2s have arrived and I've started getting the RAM upgraded and tested.
#homelab #minirack

Almost done with RAM testing, the third box is underway. Still need to Clonezilla it, but figure I can just enjoy a few beers and watch some YT videos.
#homelab

Power bricks fit perfectly in the bottom of the minirack. Might have to run a fan or two in the bottom, but we'll see how things turn out there.
#minirack #homelab

Backing up the image on the MS-A2's SSD via Clonezilla. It's so nice plugging in the IODD ST400, booting off of the Clonezilla ISO and also saving the NVMe image to the same device (the IODD ST400). It's not the fastest way of doing it, but it is very, _very_ convenient.
#homelab

New shelves came in, went ahead and pulled the shorter ones out, installed the full length ones, installed the supports, and cleaned up the power cables. Also adjusted the hooks on the back so they make a bit more sense.
Do the new shelves seem crooked? Yes. However they should support the MS-A2s a bit better.
#homelab

I now officially have all the RAM...now just waiting for the boxes to toss the RAM into.
#homelab

In one of the more painful purchases I've done in a bit: just ordered 3 kits of Crucial DDR5 RAM for the MS-A2 servers. This will take them from 32GB -> 96GB which should work out a bit better for Proxmox. Unfortunately, my wallet is crying. If everything goes well, the RAM should get here just before the servers do.
#homelab

All three nodes are installed. The "server room" is an absolute mess, but figured I'd snap a picture of the back of the minirack. I'll be cleaning up tomorrow, but for now: we are 100% back online here. Migrations work, iSCSI works...well, everything works. Very happy with how this has turned out thus far.
#minirack

Did a quick test fit of one of the Minisforum MS-A2s and ran some of the cabling. I think this should turn out pretty clean and everything fits better than expected which is a good thing. The PSUs will go in the bottom 2U "compartment" without issue which I didn't run.
#minisforum #homelab

The shelves came in, got them installed. Should be big enough to hold the MS-A2s, but if not I'll have to source the full length ones. Seems stable as well.
#minirack #homelab

Moved the switch power cables to the outside of the rack and used the red "loops" to route the cable down to the PDU. I should have enough space in the basement for the Proxmox node power bricks.
#homelab #minirack

The network cabling came in so got it installed. Not sure if this is _exactly_ the way I want to run it, but either way it's pretty clean overall. Each one of the Minisforum MS-A2s will get 2x10GbE fiber for iSCSI and 1x2.5GbE for VM traffic.
#minirack #homelab

Okay, front of the rack. Gonna need a 2U vented plate for the front but the shelves should work for the Minisforum MS-A2 nodes. Also should be enough room below that shelf for the bricks.
#minirack #homelab

Welp, after breaking out the level and getting everything perpendicular: those shelves are absolutely driving me crazy as it still looks like everything is sloped. Ended up buying the full-length T2 shelves and that should fix the problem. If not then I will resort to more drastic measures.
#minirack #homelab