Workarounds (and their tradeoffs) for integrated storage constraints

Over the Thanksgiving 2023 holiday weekend, I decided to retire my trusty silver-color early-2015 13” MacBook Pro, which was nearing software-induced obsolescence, suffering from a Bluetooth audio bug, and more generally starting to show its age performance- and other-wise. I replaced it with a “space grey” color scheme 2020 model, still Intel x86-based, which I covered in detail in one of last month’s posts.

Over the subsequent Christmas-to-New Year’s week, once again taking advantage of holiday downtime, I decided to retire my similarly long-in-use silver late-2014 Mac mini, too. Underlying motivations were similar; pending software-induced obsolescence, plus increasingly difficult-to-overlook performance shortcomings (due in no small part to the system’s “Fusion” hybrid storage configuration). Speed limitations aside, the key advantage of this merged-technology approach had been its cost-effective high capacity: a 1 TByte HDD, visible and accessible to the user, behind-the-scenes mated by the operating system to 128 GBytes of flash memory “cache”.

Its successor was again Intel-based (as with its laptop-transition precursor, the last of the x86 breed) and space grey in color; a late-2018 Mac mini:

This particular model, versus its Apple Silicon successors, was notable (as I’ve mentioned before) for its comparative abundance of back-panel I/O ports:

And this specific one was especially attractive in nearly all respects (thereby rationalizing my mid-2023 purchase of it from Woot!). It was brand new, albeit not an AppleCare Warranty candidate (instead, I bought an inexpensive extended warranty from Asurian via Woot! parent company Amazon). It was only $449 plus tax after discounts. It included the speediest-available Intel Core i7-8700B 6-core (physical; 12-core virtual via HyperThreading) 3.2 GHz CPU option, capable of boost-clocking to 4.1 GHz. And it also came with 32 GBytes of 2666 MHz DDR4 SDRAM which, being user-accessible SoDIMM-based (unlike the soldered-down memory in its predecessor), was replaceable and even further upgradeable to 64 GBytes max.

Note, however, my prior allusion to this new system not being attractive in all respects. It only included a 128 GByte integrated SSD, to be precise. And, unlike this system’s RAM (or the SSD in the late 2014 Mac mini predecessor, for that matter), its internal storage capacity wasn’t user-upgradeable. I’d figured that similar to my even earlier mid-2011 Mac mini model, I could just boot from a tethered external drive instead, and that may still be true (online research is encouraging). However, this time I decided to first try some options I’d heard about for relocating portions of my app suite and other files while keeping the original O/S build internal and intact.

I’ve subsequently endured no shortage of dead-end efforts courtesy of latest operating system limitations coupled with applications’ shortsightedness, along with experiments that functionally worked but ended up being too performance-sapping or too little capacity-freeing to be practical. However, after all the gnashing of teeth, I’ve come up with a combination of techniques that will, I think, deliver a long-term usable configuration (then again, I haven’t attempted a major operating system update yet, so don’t hold me to that prediction). I’ve learned a lot along the way, which I hope will not only be helpful to other MacOS users but, thanks to MacOS’s BSD Unix underpinnings, may also be relevant to those of you running Linux, Android, Chrome OS, and other PC and embedded Unix-based operating systems.

Let’s begin with a review of my chosen external-storage hardware. Initially, I thought I’d just tether a Thunderbolt 3 external SSD (such as the 2TB Plugable drive that I picked up from B&H Photo Video on sale a year ago for $219) to the mac Mini, and that remains a feasible option:

However, I decided to “kill two birds with one stone” by beefing up the Mac mini’s expansion capabilities in the process. Specifically, I initially planned on going with one of Satechi’s aluminum stand and hubs. The baseline-feature set one that color-matches my Mac mini’s space grey scheme has plenty of convenient-access front-panel connections, but that’s it:

Its “bigger brother” additionally supports embedding a SATA (or, more recently, NVMe) M.2 format SSD, but connectivity is the same 5-or-more-recently-10 Gbps USB-C as before (ok for tethering peripherals, not so much for directly running apps from mass storage). Plus, it only came in a silver color scheme (ok for Apple Silicon Mac minis, not so much for x86-based ones):

So, what did I end up with? I share the following photo with no shortage of chagrin:

In the middle is the Mac mini. Above it is a Windows Dev Kit 2023, aka “Project Volterra,” an Arm- (Qualcomm Snapdragon 8cx Gen 3, to be precise, two SoC steppings newer than the Gen 1 in my Surface Pro X) and Windows 11-based mini PC, which I’ll say more about in a future post.

And at the bottom of the stack is my external storage solution—dual-storage, to be precise—an OWC MiniStack STX in its original matte black color scheme (it now comes in silver, too).

Does it color-match the Mac mini? No, even putting aside the glowing blue OWC-logo orb on the front panel. And speaking of the front panel, are there any easily user-accessible expansion capabilities? Again, no. In fact, the only expansion ports offered are three more Thunderbolt 3 ones around back…the fourth there connects to the computer. But Thunderbolt 3’s 40 Gbps bandwidth is precisely what drove my decision to go with the OWC MiniStack STX, aided by the fact that I’d found a gently used one on eBay at substantial discount from MSRP.

Inside, I’ve installed a 2 TByte Samsung 980 Pro PCIe 4.0 NVMe SSD which I bought for $165.59 used at Amazon Warehouse a year ago (nowadays, new ones sell for the same price…sigh…):

alongside a 2 TByte Kingston 2.5” KC600 2.5” SATA SSD:

They appear as separate external drives on system bootup, and the performance results are nothing to sneeze at. Here’s the Samsung NVMe PCI 4.0 SSD (the enclosure’s interface to the SSD, by the way, is “only” PCIe 3.0; it’s leaving storage performance potential “on the table”):

and here’s the Kingston, predictably a bit slower due to its SATA III interface and command set (therefore rationalizing why I’ve focused my implementation attention on the Samsung so far):

For comparison, here’s the Mac mini’s internal SSD:

The Samsung holds its own from a write performance standpoint but is more than 3x slower on reads, rationalizing my strategy to keep as much content as possible on internal storage. To wit, how did I decide to proceed, after quickly realizing (mid-system setup) that I’d fill up the internal available 128 GBytes well prior to getting my full desired application suite installed?

(Abortive) Step 1: Move my entire user account to external storage

Quoting from the above linked article:

In UNIX operating systems, user accounts are stored in individual folders called the user folder. Each user gets a single folder. The user folder stores all of the files associated with each user, and settings for each user. Each user folder usually has the system name of the user. Since macOS is based on UNIX, users are stored in a similar manner. At the root level of your Mac’s Startup Disk you’ll see a number of OS-controlled folders, one of which is named Users.

Move (copy first, then delete the original afterwards) an account’s folder structure elsewhere (to external storage, in this case), then let the foundation operating system know what you’ve done, and as my experience exemplifies, you can free up quite a lot of internal storage capacity.

Keep in mind that when you relocate your user home folder, it only moves the home folder – the rest of the OS stays where it was originally.

One other note, which applies equally to other relocation stratagems I subsequently attempted, and which perhaps goes without saying…but just to cover all the bases:

Consider that when you move your home folder to an external volume, the connection to that volume must be perfectly reliable – meaning both the drive and the cable connecting the drive to your Mac. This is because the home folder is an integral part of macOS, and it expects to be able to access files stored there instantly when needed. If the connection isn’t perfectly reliable, and the volume containing the home folder disappears even for a second, strange and undefined behavior may result. You could even lose data.

That all being said, everything worked great (with the qualifier that initial system boot latency was noticeably slower than before, albeit not egregiously so), until I noticed something odd. Microsoft’s OneDrive client indicated that it has successfully sync’d all the cloud-resident information in my account, but although I could then see a local clone of the OneDrive directory structure, all of the files themselves were missing, or at least invisible.

This is, it turns out, a documented side effect of Apple’s latest scheme for handling cloud storage services. External drives that self-identify as capable of being “ejectable” can’t be used as OneDrive sync destinations (unless, perhaps, you first boot the system from them…dunno). And the OneDrive sync destination is mirrored within the user’s account directory structure. My initial response was “fine, I’ll bail on OneDrive”. It turns out, however, that Dropbox (on which I’m much more reliant) is, out of operating system support necessity, going down the same implementation-change path. Scratch that idea.

Step 2: Install applications to external storage

This one seems intuitively obvious, yes? Reality proved much more complicated and ultimately limited in its effectiveness, however. Most applications I wanted to use that had standalone installers, it turns out, didn’t even give me an option to install anywhere but internal storage. And for the ones that did give me that install-redirect option…well, please take a look at this Reddit thread I started and eventually resolved, and then return to this writeup afterwards.

Wild, huh? That said, many MacOS apps don’t have separate installer programs; you just open a DMG (disk image) file and then drag the program icon inside (behind which is the full program package) to the “Applications” folder or anywhere else you choose. This led to my next idea…

Step 3: Move already-installed applications to external storage

As previously mentioned, “hiding” behind an application’s icon is the entire package structure. Generally speaking, you can easily move that package structure intact elsewhere (to external storage, for example) and it’ll still run as before. The problem, I found out, comes when you subsequently try to update such applications, specifically where a separate updater utility is involved. Take Apple’s App Store, for example. If you download and install apps using it (which is basically the only way to accomplish this) but you then move those apps elsewhere, the App Store utility can no longer “find” them for update purposes. The same goes for Microsoft’s (sizeable, alas) Office suite. In these and other cases, ongoing use of internal storage is requisite. Conversely, apps with integrated update facilities, such as Mozilla’s Firefox and Thunderbird, or those that you update by downloading and swapping in a new full-package version, upgrade fine post-move.

Step 4: Move data files, download archives, etc. to external storage

I mentioned earlier that Mozilla’s apps (for example) are well-behaved from a relocation standpoint. I was specifically referring to the programs themselves. Both Firefox and Thunderbird also create user profiles, which by default are stored within the MacOS user account folder structure, and which can be quite sizeable. My Firefox profile, for example, is just over 3 GBytes in size (including the browser cache and other temporary files), while my Thunderbird profile is nearly 9 GBytes (I’ve been using the program for a long time, and I also access email via POP3—which downloads messages and associated file attachments to my computer—vs IMAP). Fortunately, by tweaking the entries in both programs’ profiles.ini files, I’ve managed to redirect the profiles to external storage. Both programs now launch more slowly than before, due to the aforementioned degraded external drive read performance, but they then run seemingly as speedy as before, thanks to the aforementioned comparable write performance. And given that they’re perpetually running in the background as I use the computer, the launch-time delay is a one-time annoyance at each (rare) system reboot.

Similarly, I’ve redirected my downloaded-files default (including a sizeable archive of program installers) to external storage, along with an encrypted virtual drive that’s necessary for day-job purposes. I find, in cases like these, that creating an alias from the old location to the new is a good reminder of what I’ve previously done, if I subsequently find myself scratching my head because I can’t find a particular file or folder.

The result

By doing all the above (steps 2-4, to be precise), I’ve relocated more than 200 GBytes (~233 GBytes at the moment, to be precise) of files to external storage, leaving me with nearly 25% free in my internal storage (~28 GBytes at the moment, to be precise). See what I meant when I earlier wrote that in the absence of relocation success, I’d “fill up the available 128 GBytes well prior to getting my full desired application suite installed”? I should clarify that “nearly 25% free storage” comment, by the way…it was true until I got the bright idea to command-line install recently released Wine 9, which restores MacOS compatibility (previously lost with the release of 64-bit-only MacOS 10.15 Catalina in October 2019)…which required that I first command-line install the third-party Homebrew package manager…which also involved command-line installing the Xcode Command Line Tools…all of which installed by default to internal storage, eating up ~10 GBytes (I’ll eventually reverse my steps and await a standalone, more svelte package installer for Wine 9 to hopefully come).

Thoughts on my experiments and their outcomes? Usefulness to other Unix-based systems? Anything else you want to share? Let me know in the comments!

—Brian Dipert is the Editor-in-Chief of the Edge AI and Vision Alliance, and a Senior Analyst at BDTI and Editor-in-Chief of InsideDSP, the company’s online newsletter.

<!–

VIDEO AD

–>

<!–

div-gpt-ad-inread

–>

<!–
googletag.cmd.push(function() { googletag.display(‘div-gpt-ad-native’); });
–>

The post Workarounds (and their tradeoffs) for integrated storage constraints appeared first on EDN.

Workarounds (and their tradeoffs) for integrated storage constraints

Become a member

Become a member

Become a subscriber

Become a sponsor