Almost Timely News: ๐️ How to Build a Cheap AI Setup (2026-08-09)
Almost Timely News: ๐️ How to Build a Cheap AI Setup (2026-08-09)How cheap can you make an AI setup that still works?
Almost Timely News: ๐️ How to Build a Cheap AI Setup (2026-08-09) :: View in Browser
The Big Plug๐ New merch! The Trust Insights Deep Research Suite Content Authenticity Statement100% of this week’s newsletter was made by me, the human. Learn why this kind of disclosure is a good idea and might be required for anyone doing business in any capacity with the EU in the near future. Watch This Newsletter On YouTube ๐บ
Click here for the video ๐บ version of this newsletter on YouTube » Click here for an MP3 audio ๐ง only version » What’s On My Mind: How to Build a Cheap AI SetupI’ve been putting together a new course for Trust Insights, AI for Writing (and Writers), which will be coming out August 25th, and during the process of creating that course, I realized I’ve never walked through setting up a discount agentic AI system. For a while now, many of us who work in “corporate” have had access to what feels like unlimited amounts of AI (though that free ride is coming to an end at enterprise shops thanks to absurdly large bills). For the average person who doesn’t have the backing of a corporation, how could we tap into the power of agentic systems without tapping out our wallets? This tracks back to a question from Leslie in the Analytics for Marketers Slack group: “have you done a video or post with recommendations for a cheap second computer set up someone can use to build their separate AI machine?“ So this week, let’s get scrappy. Warnings, Disclaimers, DisclosuresIf you work at a company, please check with IT before installing software willy-nilly. As a former IT professional, we thank you. Second, in this issue I’ll be talking about Chinese-made AI models like Qwen and DeepSeek. There is a HUGE difference between a Chinese model hosted by a company in the People’s Republic of China (PRC), and a Chinese-made model hosted by a company outside the PRC. It is manifestly unsafe to use any AI models hosted in the PRC for any kind of confidential information, period. Totally fine to use those models for things that are completely public, like research requests. However, it is completely safe to use Chinese-made AI models on your own hardware, or in an infrastructure that’s trustworthy. There are many, many great inference providers that have things like Zero Data Retention (ZDR) hosting which are even safer to use than Big Tech models because the providers warrant that they don’t keep your data at all. (Most Big Tech providers have a “we hold onto your data for 30 days to prevent abuse” clause in their Terms of Service) There is a lot of fear mongering about open weights models (including the Chinese ones) happening right now because Big Tech providers’ profits are on the line if these models continue to offer parity of capabilities at a fraction of the cost. Don’t believe the hype; open weights models are so safe that companies like HuggingFace have used them to do their own cybersecurity defenses. Third, where I have commercial relationships, I’ll disclose them. I have an Amazon affiliate link in this week’s article, and obviously I work for my company, Trust Insights. I do not have a financial interest in the other companies mentioned as of the time of writing. (but I’m happy to, call me) Part 1: The ModelWe’ll begin our tour with what model to use. In general, you want models that are a blend of cost efficient with effective performance. One of my favorite sites for this is Artificial Analysis; there’s a nice little grid on their models page that shows cost versus intelligence. What we’re looking for in this grid is models that are at the top left corner - the vertical (y) axis is brains, where higher is better, and the horizontal (x) axis is cost, where to the left is lower.
The current winner here is DeepSeek V4 Flash 0731, which has a crazy low cost and lots of brains, on par with OpenAI’s newest model, GPT-5.6 Luna. On the far right is Claude Opus 5, which has about 15% more brains but 40x the cost. The worst quadrant to be in is the lower right - high cost, low brains. For the purposes of building the cheapest agentic system, DeepSeek is going to be your daily driver model. Now, it’s worth pointing out, as I have in my book Almost Timeless: 48 Foundation Principles of Generative AI that a daily driver may not be the smartest model, but in terms of day to day work, a model like DeepSeek v4 Flash is going to get the job done most of the time. Here’s a bit of retrospective, by the way, putting DeepSeek on the same scale as all of OpenAI’s major models over the years to see where it stacks up in terms of brains.
DeepSeek V4 Flash outcompetes all but the most recent releases of OpenAI’s models over the years; it is almost 50% smarter than last summer’s flagship model, GPT-5, and is only 17% less capable than OpenAI’s current flagship model. Here’s the same chart from the Claude world:
DeepSeek V4 Flash is about on parity with Claude Opus 4.7 - which came out in April 2026, a scant FOUR MONTHS AGO. Now, times change, tech changes, and what you should be doing is checking in on these kinds of charts over time to see which model is closest to that top left corner. This month, it’s DeepSeek V4 Flash. Next month, it might be completely different. It’s our responsibility to keep up with the charts and use the cheapest, best brains we can get a hold of. Now that we know what model we’re aiming to use, we have to figure out where to use it. Part 2: The HostJust like web hosting companies host websites, model hosting companies - called inference providers - host models. There are so, so many of these out there, and they compete mainly on two things, speed and price. Some providers charge a premium but generate results (tokens per second) at ridiculous speeds, like Cerebras. Others offer a wide variety of models at slower speeds but are cheap (like DeepInfra). Choosing a provider is straightforward: because you’ve decided the model already, it’s a question of pricing and availability. In Google AI Mode, modify this prompt to suit your needs:
You might want to add specific additional needs like SOC 2 compliance, GDPR compliance, etc. To the prompt based on your needs, so customize it any way you like. With any model you choose, be sure you capture the full model name; DeepSeek v4 Flash for example has an antecedent that lacks the 0731 suffix. Our exact model that we want is DeepSeek v4 Flash 0731. When I stick this into Google, DeepInfra, Fireworks AI, and OpenCode Zen show up as good options. Which one you choose will be based on how much you prioritize speed versus cost. If you told me to pick one and only one, I’ve been a customer of DeepInfra for a couple of years now and have no complaints at all, BUT there is also a case to be made for OpenCode Zen. More on that in a minute. What I like, for Leslie’s specific use case, is that pay as you go providers mean that you’re not paying if you’re not using the AI AND there’s a strong economic incentive for you to use AI as little as possible and be as efficient in its use as possible. One of the problems with token plans (which include things like Claude Pro/Max, etc.) is that if you haven’t hit your weekly/monthly limits, you’re wasting money - and thus you’re more likely to use AI frivolously. With pay as you go, you only pay for what you use. Part 3: The HardwareFor a second computer setup that’s cheap, we have to think about how computers work. The graphical user interface we’re all used to on Windows or Mac or even desktop Linux, as well as on our phones and tablets, is an INCREDIBLY inefficient waste of compute resources. It takes enormous amounts of memory and processing power to draw and maintain the interface compared to pure text-based interfaces. Now, the crusty old GenX computer nerd in me says that everyone should know how to use a text-based interface (often called a command line or command prompt or terminal), but that’s not realistic. However, there is one user that LOVES text-based interfaces. Yup, you guessed it. That’s AI. AI is far, far more efficient at typing than it is using a mouse, so for our second computer as Leslie asked about, we’re going to aim for a super cheap box that can run a terminal, store things, and run an AI harness. At a price point of roughly USD 500, you won’t be running AI models directly on this hardware (hence why we searched for hosting providers in Part 2), but you’ll have a decent machine with good storage. On the shopping venue of your choice, look for what are called gaming mini-PCs. These typically will have AMD Ryzen chips, a small hard drive (typically 256 GB or 512 GB SSD), and between 16 GB and 32 GB of RAM. The cardinal rule of computing is that you want as much RAM/memory as money can buy; a slightly slower processor with more RAM is better than a fast processor with not enough RAM. RAM is like highway lanes; a Bugatti stuck in traffic on a single lane dirt road will not go as fast as a Prius on an 8 lane highway with no traffic jam. Google AI Mode is handy here as well. Modify this prompt to suit your needs, especially if there are some companies you would prefer not to do business with:
I spotted some random Chinese computer on Amazon that was selling for USD 499 with 32 GB of RAM and a 1 TB SSD (affiliate link!) that has a name that resembles what happens if you stuff Scrabble tiles in your mouth and then spit them out on the table. It’s got a 2023 chip in it, the AMD Ryzen 7730U, which is plenty fast for what we’re doing. Why these specs? You want the extra RAM and disk space so you can run multiple local applications together on the same box. For example, if you want to run Hermes Agent, chances are you’ll want it in something like a Docker container, and the RAM/disk space will REALLY help with that. Likewise, if you want to roll your own local apps, you’ll probably be running a server that has a database, maybe even a local webserver, and more memory and disk space will really help there. And technically, though it’s a miserable experience, a mini PC like this CAN do local inference with small models. Using something like llama.cpp, koboldcpp, or Unsloth Studio, you COULD theoretically serve up a small model like Gemma 4 E4B and get a modest little AI server running that can do basic things like summarization. Once you get your hardware, you’ll want to erase the operating system on it and install a version of Linux. Linux is the best choice for this kind of computer because it can run using very few system resources. I’d suggest Ubuntu Linux in LTS flavor - LTS stands for Long Term Support, and the Ubuntu Foundation warrants that a version of Ubuntu will be supported for 10 years from the launch date. The current version is 26.04 LTS, Resolute Raccoon. One thing that’s nice about Ubuntu Linux is that you can start it up with a GUI for configuration and all the comforts of modern computing, and then once everything is all square, you can shut down the GUI to free up resources, running it in text-based interface alone. Part 4: The HarnessAI models by themselves are smart but are prone to things like hallucinations. Model hosts can add some guardrails and some tuning, but the real magic happens when you put a good model in a great harness. Think of the model like the engine of a car. An engine isn’t much good by itself, and a car without an engine is a lawn ornament, doubly so if it’s on cinderblocks. But a good engine in a great car? That’s where, quite literally, the rubber meets the road. And as with cars, sometimes you can do with a good engine but not a great one if the rest of the car is nice. Unless you’re driving NASCAR or Formula 1, a Prius doesn’t need a V8 engine. Harnesses are the interfaces to AI. When you use ChatGPT on the web, OpenAI has put a harness around its models, and what you experience as ChatGPT is their tuned harness. The same is true for Gemini, for Claude, etc. - and different harnesses do different things. Claude Code, for example, has hundreds of words of specific language built into its system prompts to behave a certain way, plus a slew of deterministic rules that silently reject bad outputs from the Claude models before you ever see them as a user. For our purposes, because we’re trying to build a scrappy AI setup, we want a harness that’s free and supports multiple providers. For that, there’s no better harness than OpenCode. OpenCode is Free Open Source Software, and it comes in 2 flavors, a command line version and a GUI version. What’s really nice about it is that the GUI version is built on top of the command line version, so you don’t have to do anything differently and you can configure it in GUI mode, then use it later in command line mode with no changes at all. OpenCode also supports all the modern skills and plugins that other AI systems use, so if you see a cool skill that someone’s showing off in Claude Code, for example, you can install that same skill in OpenCode. OpenCode also offers 2 paid coding plans to go with it, OpenCode Zen which is pay as you go, and OpenCode Go, which is a subscription plan. There are some free models in OpenCode Zen but those DO collect your data and are not zero data retention. Remember in the world of AI that if you’re not paying, you and your data are the product being sold to someone else. If you want the convenience of not having to manage your AI hosting separate from your AI harness, OpenCode Zen is a great choice. If you want the lowest possible costs, then use OpenCode with DeepInfra. Part 5: Wrapping Up, the Process in a NutshellSo in a nutshell, here is the order of operations to get up and running - and you can put this list (or this entire newsletter, really) into your favorite chat-based AI tool to walk you through this process.
With those six big steps, you’re ready to go! I would suggest also installing and configuring tmux on your new box, because that gives you the ability to remote control it very easily, and if you want to get FANCY, install Tailscale on it as well, to create a VPN of your computers together so you can access your new AI box from anywhere in the world. That’s probably best for another time, though your AI of choice can walk you through that pretty easily. The key to remember here is that this machine is a self-contained little server. Once you have OpenCode installed on it and connected to your model of choice, one of the first tasks you can do is put it to work securing itself. Using your favorite deep research tools, ask for a best practices guide to securing your little computer and optimizing it for AI usage, especially with the most popular command line tools that will help AI save tokens and usage. Because you’re using a provider like DeepInfra, every request you make costs you a tiny bit of money, and the more you can reduce that, the more you save. If you deploy this system as written, you’ll spend about USD 500 on the hardware, and probably USD 5 to 10 per month in AI inference costs for moderate usage, which I define as a major coding project per week, something where your AI is working with sustained effort for about 2 hours a day. Models like DeepSeek V4 and similarly capable models are incredibly capable and dirt cheap to use. It’s worth pointing out that if you have a working computer today, you could skip the hardware purchase and install OpenCode on your computer as is, eliminating the hardware cost. For folks on limited incomes, this is a great way to get access to high end AI without high end subscriptions, like job seekers without stable income. Enjoy running your own little AI operations center! How Was This Issue?Rate this week’s newsletter issue with a single click/tap. Your feedback over time helps me figure out what content to create for you. Got More Feedback?๐ Please take my 6-question Reader Survey to tell me what you want! Here’s The UnsubscribeIt took me a while to find a convenient way to link it up, but here’s how to get to the unsubscribe.
If you don’t see anything, here’s the text link to copy and paste: https://almosttimely.substack.com/action/disable_email Share With a Friend or ColleaguePlease share this newsletter with two other people. Send this URL to your friends/colleagues: https://www.christopherspenn.com/newsletter For enrolled subscribers on Substack, there are referral rewards if you refer 100, 200, or 300 other readers. Visit the Leaderboard here. ICYMI: In Case You Missed ItHere’s content from the last week in case things fell through the cracks:
My Merch ShopI’ve been adding so much stuff that I’ve decided to bundle it all in what I call a Merch Shop, because otherwise there’s literally too much to keep track of and I run out of space in my own newsletter. So welcome to the Merch Shop! Books: Skills for Claude and Agentic AI:
Courses: Subscriptions: On The TubesHere’s what debuted on my YouTube channel this week:
Advertisement: New GEO 201 CourseIn GEO 101, the first course I built on the basics of GEO, I taught you about presence, appearance, and relevance, the three phases of GEO, and what you need to do in each phase to align with how AI search operates. The top piece of feedback we got at Trust Insights about it was, “okay, great, but how do I tell my boss that we’re ‘winning’ at GEO?“ After I quelled my murderous rage at your boss on your behalf, Katie and I sat down and worked out a straightforward, aligned methodology for doing this. GEO 201 is based on the three phases, what you can control and what you can genuinely see - and critically, what you can’t. Because there is absolutely no way to say your brand “ranks higher” in AI search, period, end of story. But you can say and show with confidence what you’ve done and how you show up for presence, appearance, and relevance with tools you’re probably already paying for, and based on how AI search systems really work. ๐ GEO 201 is available now for USD 149. Get Back To Work!Folks who post jobs in the free Analytics for Marketers Slack community may have those jobs shared here, too. If you’re looking for work, check out these recent open positions, and check out the Slack group for the comprehensive list. Disclosure: I source these links from LinkedIn every week on the following criteria: New in the past seven days, Easy Apply on, remote roles, USA geography. How to Stay in TouchLet’s make sure we’re connected in the places it suits you best. Here’s where you can find different content:
Listen to my theme song as a new single: Social Good: Ukraine ๐บ๐ฆ Humanitarian FundThe war to free Ukraine continues. If you’d like to support humanitarian efforts in Ukraine, the Ukrainian government has set up a special portal, United24, to help make contributing easy. The effort to free Ukraine from Russia’s illegal invasion needs your ongoing support. ๐ Donate today to the Ukraine Humanitarian Relief Fund » Events I’ll Be AtHere are the public events where I’m speaking and attending. Say hi if you’re at an event also:
There are also private events that aren’t open to the public. If you’re an event organizer, let me help your event shine. Visit my speaking page for more details. Can’t be at an event? Stop by my private Slack group instead, Analytics for Marketers. Required DisclosuresEvents with links have purchased sponsorships in this newsletter and as a result, I receive direct financial compensation for promoting them. Advertisements in this newsletter have paid to be promoted, and as a result, I receive direct financial compensation for promoting them. My company, Trust Insights, maintains business partnerships with companies including, but not limited to, Amazon, Talkwalker, MarketingProfs, Agorapulse, The Marketing AI Institute, Spin Sucks, and others. While links shared from partners are not explicit endorsements, nor do they directly financially benefit Trust Insights, a commercial relationship exists for which Trust Insights may receive indirect financial benefit, and thus I may receive indirect financial benefit from them as well. Thank YouThanks for subscribing and reading this far. I appreciate it. As always, thank you for your support, your attention, and your kindness. Please share this newsletter with two other people. See you next week, Christopher S. Penn Invite your friends and earn rewards
If you enjoy Almost Timely Newsletter, share it with your friends and earn rewards when they subscribe.
|
Comments