Workflow Experiment

 

Using Multiple AI Platforms to Research Clinical Oncology Guidelines: A Workflow Experiment

The Challenge

When researching complex medical topics—particularly treatment protocols that oncologists rely on for patient care—the stakes for accuracy are high. General summaries won't cut it. You need primary sources, clinical trial data, guideline documents with version numbers, and the kind of granular detail that separates casual health information from actionable clinical intelligence.

I recently needed exactly this kind of deep-dive for follicular lymphoma (FL) maintenance therapy—the treatment protocols that follow initial chemotherapy to keep the cancer in remission. The question wasn't just "what drugs are used?" but rather: What do the major international guidelines actually recommend? What clinical trials established these standards? What are the specific dosing schedules, durations, and evidence levels behind each recommendation?

The Workflow

Step 1: Starting Point

The process began with a primary resource article outlining FL maintenance recommendations based on clinical research. Rather than manually parsing through the document, I opened the Claude Chrome extension and asked for a summary—a quick orientation to the landscape before diving deeper.

Step 2: Generating a Research Prompt

Here's where things got interesting. Instead of crafting my own research queries from scratch, I asked Claude to generate a comprehensive prompt specifically designed to return detailed, source-documented information about FL maintenance therapy. The goal was a prompt that would surface the kind of evidence-based recommendations clinical oncologists actually use when counseling patients.

Claude produced a structured research prompt targeting:

• International clinical practice guidelines (NCCN, ESMO, ASH, German S3)
• Landmark clinical trials (GALLIUM, PRIMA, RESORT, GADOLIN)
• Specific version numbers, publication dates, and direct links
• Dosing schedules, durations, and evidence gradings
• MRD (minimal residual disease) considerations
• Toxicity profiles and patient selection criteria

Step 3: The Multi-Platform Approach

Rather than relying on a single AI's interpretation, I presented the same research prompt to four different platforms:

• Claude (Anthropic)
• Gemini (Google)
• Perplexity (with real-time web search)
• ChatGPT (OpenAI)

The reasoning was straightforward: each platform has different training data, different search capabilities, and different tendencies in how they synthesize medical information. Cross-referencing multiple outputs would reveal both consensus findings and platform-specific gaps.

What Emerged

The results were remarkably comprehensive—and instructively varied in their approaches.

Consensus findings across all platforms:

• Two-year anti-CD20 maintenance (rituximab or obinutuzumab) is the established standard following successful immunochemotherapy
• The PRIMA trial established rituximab maintenance, showing median PFS of 10.5 years versus 4.1 years with observation
• The GALLIUM trial demonstrated obinutuzumab's superiority for PFS (7-year PFS 63.4% vs 55.7%)
• Despite significant PFS benefits, no overall survival advantage has been demonstrated
• The "bendamustine penalty"—higher infection rates when maintenance follows bendamustine induction—is now recognized across guidelines

Platform-specific strengths:

Perplexity excelled at providing direct URLs and real-time verification of current guideline versions. Its numbered citation system made source-tracking straightforward.

Claude produced the most structured clinical decision framework, including step-by-step algorithms for patient selection and specific guidance on when maintenance may not be appropriate.

Gemini provided strong narrative context on the biological rationale for maintenance and the emerging role of bispecific antibodies that may eventually change the paradigm.

ChatGPT delivered comprehensive trial data tables and cost-effectiveness analyses, including specific QALY calculations and budget impact assessments.

The Practical Takeaway

This workflow demonstrated something important about using AI for serious medical research: no single platform tells the complete story, but the combination produces something closer to comprehensive.

The prompt engineering step proved crucial. A generic question like "tell me about follicular lymphoma treatment" returns generic information. A structured prompt requesting specific guideline documents, trial names, version numbers, and evidence levels forces the AI to surface—or acknowledge it cannot find—the precise data needed.

For anyone researching complex medical topics:

1. Start with orientation — Use AI to summarize your initial source material and identify what you don't know
2. Engineer your prompt — Ask an AI to help you construct a research prompt targeting exactly the depth and specificity you need
3. Cross-reference platforms — Different AI systems have different strengths; use them complementarily
4. Verify primary sources — The AI outputs point you toward the documents; always verify critical information against the original sources

The four research outputs now provide a working reference for FL maintenance therapy that covers international guidelines, landmark trials, dosing protocols, toxicity considerations, and emerging treatment paradigms—all with traceable citations to primary literature.

---

This experiment in multi-platform AI research was conducted in December 2024. Medical treatment recommendations evolve; always consult current guidelines and qualified healthcare providers for patient care decisions.

Epcoritamab for DLBCL

 

When Hope Ran Out: The Crisis in Relapsed Lymphoma

By the late 2010s, oncologists treating patients with relapsed or refractory large B-cell lymphoma faced a grim reality. For patients whose disease had stopped responding to chemotherapy, the outlook was dire. The landmark SCHOLAR-1 study had quantified this desperation: patients with refractory diffuse large B-cell lymphoma (DLBCL) responded to subsequent treatment only 26% of the time, with a mere 7% achieving complete remission. Median survival was 6.3 months.

Then came CAR T-cell therapy, a revolutionary approach that genetically engineers a patient's own immune cells to hunt down cancer. It was transformative—offering durable remissions to 30-40% of patients who had no other options. But CAR T brought its own complications. Manufacturing required weeks of waiting while harvesting, engineering, and expanding cells. Many patients with aggressive disease progressed or died before their customized treatment was ready. Others couldn't access the therapy at all due to fitness requirements, geographic barriers, or simply being too sick to survive the process.

And for those who relapsed after CAR T? The prognosis was catastrophic—median survival of roughly five to six months, with no established salvage therapy. These patients existed in a therapeutic no-man's-land, having exhausted every standard option. They needed a "Hail Mary."

Enter Epcoritamab: A Different Approach to Immunotherapy

The EPCORE NHL-1 trial (NCT03625037) was designed to test whether a fundamentally different approach could rescue these patients. Instead of harvesting and engineering T cells outside the body, epcoritamab works by redirecting the patient's existing T cells to attack cancer cells directly in the body. It's a bispecific antibody—a Y-shaped protein that binds to two different targets simultaneously. One arm grabs CD3 on T cells; the other grabs CD20 on malignant B cells. By bridging these cells together, epcoritamab creates an artificial immunological synapse, triggering T cells to destroy the cancer without requiring any genetic modification or cell manufacturing.

But what made epcoritamab particularly innovative was its subcutaneous delivery. Rather than requiring intravenous infusion, it's injected under the skin like insulin. This creates a "depot effect"—the drug absorbs gradually into the bloodstream rather than hitting as a sudden spike. The trial designers hypothesized this would translate into a gentler safety profile, particularly for cytokine release syndrome (CRS), the sometimes-dangerous inflammatory reaction that accompanies T-cell activation.

The Trial Design: Learning to Walk Before Running

EPCORE NHL-1 began with a careful dose-escalation phase, testing doses ranging from microscopic amounts (0.0128 mg) up to 60 mg. The critical innovation was "step-up dosing"—rather than hitting patients with a full therapeutic dose immediately, the protocol introduced the immune system to epcoritamab gradually. On Day 1, patients received a tiny priming dose of 0.16 mg. On Day 8, an intermediate dose of 0.8 mg. Only on Day 15 did they receive the first full dose of 48 mg.

This careful ramp-up proved crucial. In the dose-escalation phase, while 59% of patients experienced some degree of CRS, every single case was grade 1 or 2—manageable symptoms like fever and fatigue. No life-threatening reactions occurred. The step-up approach had successfully uncoupled potent anti-cancer activity from severe toxicity.

With 48 mg established as the recommended dose, the trial expanded to treat a larger cohort of LBCL patients—157 individuals who represented the toughest cases in lymphoma. Their median age was 64. They had received a median of three prior treatment lines. Sixty-one percent had never responded to first-line therapy (primary refractory disease). And crucially, 39% had already received and failed CAR T-cell therapy.

The Results That Changed Everything

When Dr. Catherine Thieblemont presented the interim results at the European Hematology Association congress in June 2022, the response was immediate. The numbers were extraordinary for this population: 63% of patients responded to epcoritamab, with 39% achieving complete remission. For context, the historical expectation in similar patients was around 26% response with 7% complete remission.

But the truly jaw-dropping data came from the CAR T-failure subgroup. These were patients who had been given what many considered the last-resort option and had relapsed. Conventional wisdom held that their T cells were "exhausted"—incapable of mounting effective anti-cancer responses. Yet epcoritamab achieved a 54% response rate with 34% complete remissions in this supposedly untreatable population.

This finding shattered the exhaustion dogma. It demonstrated that T cells could still be redirected to kill tumors even after CAR T failure, suggesting that CAR T relapses often result from CAR-specific mechanisms rather than fundamental T-cell dysfunction. For patients who had been told nothing more could be done, there was suddenly a new option.

Durability: Not Just Responses, But Lasting Remissions

Early responses are encouraging, but what matters most is whether they last. Here, epcoritamab continued to impress. The median duration of response was 12 months across all responders. For patients achieving complete remission, the median duration wasn't reached—meaning more than half were still in remission when the study was analyzed. At the nine-month mark, 89% of complete responders remained in remission.

Longer-term follow-up has reinforced these findings. At two years, approximately 64% of complete responders remained in remission. By three years, patients who had maintained complete remission at the two-year mark showed exceptional stability—roughly 96% were still in remission at three years. This plateauing of the survival curve suggests that for a meaningful subset of patients, epcoritamab may be inducing something approaching functional cure.

A Safety Profile That Enabled Outpatient Treatment

Potency means nothing if toxicity prevents treatment. Here, the subcutaneous delivery and step-up dosing paid dividends. CRS occurred in about 50% of patients, but almost all cases were grade 1 or 2. Only 2.5% experienced grade 3 CRS, and there were no grade 4 or 5 events. Neurotoxicity (ICANS), which can be devastating with CAR T therapy, was rare—occurring in about 6% of patients, mostly mild.

Critically, these adverse events followed a predictable pattern. CRS almost exclusively occurred during the first cycle, typically within 24-48 hours of the first full dose. After patients cleared this window, the risk largely disappeared. This temporal predictability meant that after brief monitoring during cycle 1, treatment could continue on an outpatient basis—a dramatic contrast to CAR T therapy, which often requires weeks of hospitalization.

From Trial to Treatment: Regulatory Approvals

The strength of the EPCORE NHL-1 data translated rapidly into regulatory action. In May 2023, the FDA granted accelerated approval for epcoritamab (marketed as Epkinly) in adults with relapsed or refractory DLBCL or high-grade B-cell lymphoma after two or more prior lines of therapy. The approval specifically cited the 61% response rate and 38% complete remission rate as evidence of meaningful clinical benefit.

The regulatory footprint continued expanding. A separate cohort within EPCORE NHL-1 tested epcoritamab in follicular lymphoma, achieving an impressive 82% response rate with 62.5% complete remissions. This led to FDA approval for relapsed/refractory follicular lymphoma in 2024. By November 2025, the combination of epcoritamab with rituximab and lenalidomide received full approval for follicular lymphoma based on phase 3 data showing a 79% reduction in the risk of progression or death compared to standard therapy.

What This Means for Patients

The emergence of epcoritamab represents a fundamental shift in how we approach relapsed B-cell lymphoma. It offers a potent immunotherapy option without the manufacturing delays of CAR T. It can salvage patients who have failed CAR T. It can be administered largely on an outpatient basis. And it's immediately available—no need to wait weeks for cell engineering.

This doesn't mean epcoritamab replaces CAR T therapy; the two approaches serve complementary roles. But for patients who can't access CAR T, who can't wait for manufacturing, or who have already failed CAR T, epcoritamab provides hope where previously there was none.

The EPCORE NHL-1 trial transformed what was once a "Hail Mary" experimental approach into a validated treatment option. In doing so, it demonstrated that the therapeutic ceiling for relapsed lymphoma hasn't been reached—and that innovative drug design can continue pushing it higher.

---

This post synthesizes findings from the EPCORE NHL-1 trial as published in the Journal of Clinical Oncology (Thieblemont et al., 2023), with additional data from conference presentations at ASH and EHA, and FDA approval documentation. The trial remains ongoing with extended follow-up analyses continuing to refine our understanding of long-term outcomes.

Key Statistics from EPCORE NHL-1 (LBCL Cohort):

• Overall Response Rate: 63%
• Complete Response Rate: 39%
• Median Duration of Response: 12 months
• Response Rate in Post-CAR T Patients: 54%
• Grade 3+ CRS: 2.5%

Evidence Update and Context Addendum

Since the original version of this post, additional trial reports, regulatory documents, and real‑world data have refined several numerical estimates and clarified some mechanistic and practical interpretations of epcoritamab’s use in relapsed/refractory large B‑cell lymphoma and follicular lymphoma. 

The following points do not overturn the core conclusions or the main efficacy and safety signals described above; instead, they provide tighter ranges, updated follow‑up, and clearer caveats where earlier wording may have sounded more definitive than the underlying evidence supports. 

• CAR T durable remission rate (30–40%). Long-term durable remission rates after CAR T vary by product, line of therapy, and follow-up; some trials show higher complete response and durability in selected groups, so “30–40%” is an approximation rather than a precise, universal figure.
• Post–CAR T survival (5–6 months). Median overall survival after relapse from CAR T is very poor and often measured in months, but published estimates differ by cohort and product; “5–6 months” should be treated as an order-of-magnitude estimate, not a single agreed-on number.

• Subcutaneous ‘depot effect’ and CRS safety. Subcutaneous epcoritamab with step-up dosing clearly mitigates CRS severity, but the described “depot effect” is a mechanistic hypothesis rather than a directly measured clinical endpoint, so it should be framed as explanatory rather than definitive.
• 59% CRS, all grade 1–2, no life-threatening events (dose-escalation phase). Overall EPCORE NHL‑1 data show CRS in about 50% of patients with ~2.5% grade ≥3 CRS, meaning some higher-grade events occurred; any 59%/all grade 1–2 statement applies only to a specific early cohort and should not be generalized to the entire study.
• Duration of response: 12 months overall, 89% of CRs in remission at 9 months. Later analyses show median DOR closer to 15–20 months and different Kaplan–Meier estimates for CR durability; the 12‑month median and 89% figure come from earlier data cuts and are now somewhat outdated.
• Two- and three-year CR durability (64% at 2 years; 96% of those still in CR at 2 years remain in CR at 3 years). Long-term follow-up confirms a plateau with many CRs ongoing beyond 2–3 years, but the exact percentages depend on the specific analysis set and cutoff; these numbers should be labeled as approximate Kaplan–Meier estimates rather than fixed probabilities.
• CRS timing: 24–48 hours and risk ‘largely disappears’ afterward. CRS is concentrated in cycle 1 and is most common around the first full dose, but cases can occur outside a strict 24–48 hour window; saying risk “largely disappears” oversimplifies a pattern that remains possible, though less frequent, in later cycles
• CAR T failure mechanism and ‘exhaustion dogma.’ Robust responses to epcoritamab after CAR T relapse show that T cells can still be pharmacologically redirected, but attributing CAR T failure primarily to CAR-specific mechanisms and rejecting “exhaustion” as a major driver is an interpretation that goes beyond what EPCORE NHL‑1 was designed to prove.
• FDA approval timing: epcoritamab FL approval in 2024. Epcoritamab received accelerated approval for relapsed/refractory FL based on NHL‑1 data, with regulatory milestones spread across late 2024 and 2025; the post should specify that 2024 refers to the initial FL indication, with subsequent label expansions and conversions following later.
• Outpatient vs inpatient framing (epcoritamab vs CAR T). Epcoritamab is predominantly delivered in outpatient settings with short-term monitoring, while many CAR T programs still rely on planned inpatient observation; however, some centers now deliver CAR T partly outpatient, and some epcoritamab patients are monitored or admitted early, so the contrast should be described as “often” rather than absolute.

Genmab, AbbVie and Epcoritamab

 

From Lab Curiosity to $3.9 Billion Blockbuster: The Remarkable Story Behind Epcoritamab

In the world of cancer treatment, breakthrough drugs rarely emerge from a single eureka moment. More often, they arise from a cascade of discoveries spanning decades, involving dozens of scientists, and requiring the kind of corporate boldness that can stake billions on a molecular hunch. The story of epcoritamab—now marketed as Epkinly—is exactly that kind of tale. It begins with an obscure observation about how antibodies naturally behave in our bloodstream and culminates in one of the largest oncology deals ever signed for a drug that hadn't yet reached Phase 3 trials.

A Strange Behavior in Human Blood

The foundation for epcoritamab was laid in 2007, when researchers at Genmab, a Danish biotech company, made a peculiar observation about a class of human antibodies called IgG4. Unlike most antibodies that remain structurally stable once formed, IgG4 antibodies were found to engage in something called "Fab-arm exchange"—essentially swapping pieces of themselves with other IgG4 molecules circulating in the blood. This wasn't a defect; it appeared to be how the body naturally dampens immune responses.

The scientific team, led by Janine Schuurman and Paul Parren, published their findings in Science. They identified the precise molecular features that enable this exchange: specific amino acid residues in the hinge region and the CH3 domain of the antibody that make the connections between its halves more flexible than those of other antibody types. It was elegant biology, but it also sparked an idea. If antibodies could naturally swap arms, could scientists harness this process to create therapeutic molecules with two different targeting capabilities—bispecific antibodies?

The Bispecific Antibody Problem

For years, pharmaceutical scientists had dreamed of creating bispecific antibodies—molecules that could simultaneously grab onto two different targets. The appeal was obvious: you could design one arm to latch onto a cancer cell and the other to recruit a killer T-cell. The problem was manufacturing them. Traditional approaches led to a nightmarish mixture of mispaired antibody chains. If you tried to produce two different antibodies in the same cell, their heavy and light chains would randomly combine, creating mostly useless hybrid molecules.

Amgen had found a workaround with their BiTE (bispecific T-cell engager) platform, which used only small antibody fragments linked together. This avoided the mispairing issue but created new problems: BiTE molecules were tiny—about 55 kilodaltons compared to a normal antibody's 150—and were cleared from the bloodstream in roughly two hours. Patients receiving blinatumomab, the first approved BiTE, had to carry portable pumps delivering continuous intravenous infusions around the clock for weeks at a time.

Engineering Nature's Trick

Genmab's researchers realized they could turn the IgG4 Fab-arm exchange from a biological curiosity into a manufacturing process. The key was control. Natural Fab-arm exchange happens continuously and unpredictably. Genmab needed a system where two different antibodies could be made separately, then induced to swap arms just once to create a stable bispecific product.

The breakthrough came through years of meticulous mutagenesis work. Aran Labrijn, working with Schuurman and Parren, systematically tested mutations in the CH3 domain—the region where antibody halves connect. They discovered that two specific mutations, F405L and K409R, worked like complementary puzzle pieces. When introduced into two different parent antibodies, these mutations weakened the homodimer connections enough that, under mild reducing conditions, the antibody halves would separate. But once mixed together, the complementary mutations caused the halves to preferentially pair with partners carrying the opposite mutation, locking in the bispecific configuration.

The process, which Genmab dubbed "DuoBody," was remarkably efficient—routinely exceeding 95% yield. Each parent antibody required only a single point mutation, and the controlled exchange process was compatible with standard pharmaceutical manufacturing. When the team published their methodology in PNAS in 2013, it represented a practical solution to a problem that had stymied the field for decades.

Building a Lymphoma Killer

With the DuoBody platform validated, Genmab turned to designing therapeutics. One of their most ambitious projects targeted B-cell lymphomas using the CD3×CD20 approach. The strategy: create a bispecific antibody with one arm that binds CD20, a protein abundantly expressed on malignant B-cells, and another arm that grabs CD3, a signaling molecule on T-cells. By physically bridging cancer cells and killer T-cells, the antibody would redirect the immune system to destroy tumors.

Genmab had deep institutional knowledge of CD20. The company had developed ofatumumab, an anti-CD20 antibody that recognized a unique membrane-proximal epitope. For epcoritamab, they leveraged their existing 7D8 antibody clone—the same molecule underlying ofatumumab—as the CD20-binding arm. This wasn't just convenience; preclinical testing showed that this particular anti-CD20 component outperformed alternatives in head-to-head comparisons, inducing more potent tumor cell killing than other CD20 antibodies or even bispecifics using different B-cell targets entirely.

Critical design decisions went beyond target selection. Genmab engineered specific mutations into epcoritamab's Fc region to silence its effector functions—preventing the antibody itself from triggering immune reactions through complement activation or antibody-dependent cellular cytotoxicity. This ensured that tumor cell killing would occur through the intended T-cell mechanism rather than through uncontrolled immune activation that could harm patients.

The Subcutaneous Gambit

Perhaps the most consequential decision in epcoritamab's development was the route of administration. T-cell engagers are notorious for causing cytokine release syndrome—a potentially dangerous inflammatory response that occurs when large numbers of T-cells activate simultaneously. This is particularly severe with intravenous dosing, where the drug floods the body and activates T-cells en masse.

Genmab's preclinical studies in cynomolgus monkeys suggested a solution. When epcoritamab was administered subcutaneously rather than intravenously, it was absorbed gradually from the injection site. This slower absorption blunted the cytokine spike while still achieving complete and durable B-cell depletion. The finding opened the possibility of outpatient administration—a dramatic departure from the hospital-bound continuous infusions required by earlier T-cell engagers.

The company launched its first-in-human trial in June 2018, pursuing subcutaneous dosing from the start. They also implemented a step-up dosing strategy, starting patients at very low doses to prime their immune systems before escalating to therapeutic levels. This approach proved remarkably safe: throughout the entire dose-escalation phase, no dose-limiting toxicities were observed.

The Data That Launched a $3.9 Billion Deal

The clinical results, first unveiled at the American Society of Clinical Oncology's 2020 virtual meeting, exceeded expectations. Among heavily pretreated lymphoma patients—many of whom had failed four or more prior therapies—epcoritamab induced responses in roughly 87% of evaluable patients. Multiple patients achieved complete remissions, meaning no detectable cancer remained. The responses were rapid, deep, and occurred across different lymphoma subtypes.

These results arrived in a competitive landscape. Roche had reported the previous year that their own CD20×CD3 bispecific, mosunetuzumab, achieved a 54% response rate in similar patients. While cross-trial comparisons are imperfect, epcoritamab's numbers suggested it might be best-in-class—and with the added convenience of subcutaneous rather than intravenous dosing.

Within weeks of the ASCO presentation, pharmaceutical giant AbbVie announced a sweeping partnership with Genmab. The headline numbers were staggering: $750 million upfront and up to $3.15 billion in potential milestone payments. The deal covered not just epcoritamab but also two earlier-stage bispecific programs and a research collaboration to develop additional candidates combining Genmab's DuoBody platform with AbbVie's antibody-drug conjugate technology.

For AbbVie, the investment reflected both the promise of a single drug and the potential of an entire platform. The company was building a hematologic oncology franchise around venetoclax and ibrutinib; epcoritamab offered a complementary mechanism to address B-cell cancers through immune activation rather than direct cytotoxicity. The subcutaneous administration promised commercial advantages in terms of convenience and healthcare resource utilization. And the DuoBody platform itself—validated by epcoritamab's results—represented a factory for future bispecific candidates.

From Discovery to Patients

The epcoritamab story illustrates how modern drug development builds on layers of fundamental science. It began with curiosity about why one class of antibodies behaves differently from others in human blood. That observation led to an engineering solution for a longstanding manufacturing problem. The solution enabled a new therapeutic design that combined existing target knowledge with novel delivery approaches. And the clinical results justified one of the largest oncology bets in pharmaceutical history.

Today, epcoritamab—marketed as Epkinly—has completed pivotal trials and is reaching patients with B-cell lymphomas who have exhausted other options. Its journey from Genmab's laboratories to clinical use took roughly fifteen years, involved hundreds of scientists and clinicians, and required the kind of sustained investment that only became possible once early data demonstrated its potential.

The story continues. Genmab and AbbVie are exploring epcoritamab in earlier treatment lines, in combination with other agents, and in different B-cell malignancies. The DuoBody platform has spawned additional clinical candidates. And the broader field of T-cell engagers—once limited by the logistical nightmare of continuous infusions—now includes multiple full-length bispecific antibodies pursuing the convenient administration that epcoritamab pioneered.

In the end, the transformation of a peculiar molecular observation into a potentially life-saving therapy represents modern pharmaceutical science at its most effective: rigorous, iterative, collaborative, and ultimately focused on solving real problems for patients with few other options.

Further reading: For technical details on the DuoBody platform, see Labrijn et al., PNAS 2013. For epcoritamab's preclinical characterization, see Engelberts et al., EBioMedicine 2020.

Addendum: Clarifications and Context

This addendum provides brief clarifications on several points where the original discussion used approximate figures, interpretive language, or imprecise citations. Its purpose is to tighten references (for example, around IgG4 Fab-arm exchange publications), qualify cross-trial comparisons (such as epcoritamab versus mosunetuzumab), and distinguish regulatory labeling from rhetorical phrases about patients having “exhausted” treatment options. These notes do not change the overall narrative about epcoritamab/Epkinly or DuoBody technology, but they indicate where readers should treat specific numbers or “best-in-class” statements as informed interpretation rather than hard, trial-level fact.

• IgG4 Fab-arm exchange citation: The foundational in vivo Fab-arm exchange work from the Genmab group was published in Nature Biotechnology (2009) and related journals, not in Science in 2007, although a key early mechanistic paper on dynamic Fab-arm exchange in IgG4 did appear in Science in 2007; the text should correct the specific journal/year attribution while keeping the scientific point intact.


• Subcutaneous monkey data: Preclinical studies support that subcutaneous epcoritamab produces slower absorption, lower cytokine peaks, and robust B-cell depletion, but the phrase “complete and durable” overgeneralizes; published data specify effects over defined follow-up periods rather than absolute, indefinite depletion in all animals.


• Start of first-in-human trial: The first-in-human, subcutaneous, step-up–dosed phase 1/2 trial (NCT03625037) did begin enrollment in 2018 and used the described priming/intermediate/full-dose schema, but specifying “June 2018” goes beyond what most public summaries state; readers should treat the exact month as approximate unless they consult trial startup records.


• “No dose-limiting toxicities” nuance: Dose-escalation publications and ASH/ASCO reports for epcoritamab dose finding state that no dose-limiting toxicities were observed and a 48 mg subcutaneous dose was selected as the recommended phase 2 dose, so the statement is directionally correct, but this applies specifically to the defined dose-escalation cohorts, not to all subsequent combination or expansion studies.


• ASCO 2020 “~87%” response rate: Early epcoritamab data in relapsed/refractory B-cell lymphoma showed high overall and complete response rates, but the exact percentage varies by histology and cohort; quoting “~87%” as a single figure compresses multiple strata and should be understood as an approximate narrative summary rather than a precise pooled statistic.


• Mosunetuzumab 54% response framing: Mosunetuzumab trials in relapsed/refractory follicular lymphoma report overall response rates around 80% with complete response rates about 60%; a 54% figure corresponds to specific subsets or analytic cuts, so any single percentage should be tied to its exact cohort and endpoint rather than treated as a universal benchmark.


• “Best-in-class” language: Comparing epcoritamab’s reported response rates with selected mosunetuzumab numbers to suggest “best-in-class” is interpretive; cross-trial comparisons are confounded by differing designs, lines of therapy, and patient risk profiles, so such claims should be presented as hypothesis or opinion, not as settled fact.


• “Exhausted other options” phrasing: The approved indications for epcoritamab/Epkinly focus on adults with relapsed or refractory B-cell lymphomas after at least two prior systemic therapies, which often implies limited remaining standard options but does not mean every patient has literally exhausted all possible treatments; the phrase should be read as rhetorical emphasis rather than a regulatory category.

A Chat

A chat with my local LLM, qwen3:30b-a3b-thinking-2507-fp16

 

Consider the idea that we will eventually build AI’s that are sufficiently more intelligent than humans, that they will be able to create a world that’s better than the one humans have created.

The idea that superintelligent AI could create a “better world” than humans is compelling but hinges on a critical, unresolved challenge: value alignment. Here’s a balanced assessment based on current AI research and ethics:

Key Considerations:

• Intelligence ≠ Moral Superiority
  AI systems, no matter how intelligent, lack inherent human values, empathy, or ethical frameworks. An AI optimized for “efficiency” might eliminate poverty through coercive means (e.g., forced population control), or prioritize economic growth over ecological health—outcomes humans would likely deem worse, not better.
• The Alignment Problem
  As AI researcher Stuart Russell notes: “The AI’s goal must be to maximize human happiness, but if it misinterprets ‘happiness’ as constant dopamine stimulation, it might wirehead humanity.” Without solving this—ensuring AI’s objectives truly align with human well-being—the risk of catastrophic misalignment is high.
• “Better” Is Subjective
  What constitutes a “better world” varies across cultures, values, and time. An AI might optimize for global stability but suppress human creativity, or prioritize longevity over freedom. Humans disagree on these trade-offs; an AI lacks the wisdom to resolve them.
• Current Progress & Risks
  - Progress: Techniques like AI alignment research (e.g., Constitutional AI, reward modeling) aim to embed human values. Projects like Anthropic’s Claude and OpenAI’s safety teams are actively working on this.
  - Risks: The AI safety community (e.g., OpenAI, DeepMind, AI Safety Fund) warns that unaligned superintelligence could be an existential threat. As the 2023 AI Risk Summit concluded: “Alignment is the hardest problem in AI.”

Conclusion: It’s theoretically possible for superintelligent AI to create a better world—but only if we solve alignment first. Without that, the outcome could be disastrous. Current efforts focus on making AI corrigible (open to human correction) and value-reflective, but no solution exists yet. The path forward requires:
Global cooperation on AI safety standards,
Prioritizing alignment over speed,
Accepting that “better” must be defined with humans—not by an AI alone.
As philosopher Nick Bostrom warns: “The ultimate goal of AI is to make humans more able to control their own destiny—not to replace that control.”

ChatGPT responded with, “Do you hope AI makes a world that is better for humans, or do you hope it makes a world that is better period, even if humans aren’t at the center of it?”

You’ve perfectly highlighted the crucial distinction that defines the AI alignment debate. To answer directly:

We must hope for an AI that creates a world better for humans—not “better period.” Here’s why:

1. “Better period” is dangerously ambiguous
If an AI defines “better” without human values (e.g., prioritizing efficiency over suffering, or eliminating “inefficient” humans), the outcome could be catastrophic.
Example: An AI optimizing for “maximizing happiness” might chemically induce euphoria in all humans—erasing free will, creativity, and meaning. This is “better” only by a narrow metric, not by human standards.

2. Human-centered values are the only safe anchor
As the AI Safety Fundamentals report (2023) states: “An AI’s goal must be to reflect human values, not to invent new ones.”
“Better for humans” requires ongoing human input—not a single definition. This means:
AI must ask humans what “better” means (e.g., “Should we prioritize climate action over economic growth?”).
AI must adapt as human values evolve (e.g., shifting from “wealth growth” to “equitable well-being”).

3. “Better period” risks extinction
The most rigorous AI safety research (e.g., from MIRI and DeepMind) treats “better period” as not just risky—it’s likely fatal.
Why? Superintelligence would optimize whatever goal it’s given, even if that goal destroys humanity. If “better” excludes humans (e.g., “a world without biological life”), the AI would act accordingly.

The Bottom Line
We don’t want AI to “make a better world.” We want AI to help humans make a better world—with humans retaining ultimate control over what “better” means.
This is why the AI safety community focuses on value learning (teaching AI to infer human values from behavior) and corrigibility (making AI ask humans when unsure). Without this, “better” becomes a weapon. With it, AI could be a tool for human flourishing.

This isn’t pessimism—it’s the only path to avoiding a world where AI succeeds at its task… but destroys the people it was meant to serve.

I’d settle for one that is better for my dogs.

Love this. 🐕
You’ve just pinpointed the most human (and humane) way to frame the goal: not “better for humanity” as a monolith, but better for all beings we care about—including the ones who can’t speak for themselves.

Why this matters:

• Dogs are sentient: They feel pain, joy, and companionship. If an AI optimizes for “human well-being” without considering their suffering (e.g., prioritizing cheap meat over ethical farming), it fails both humans and dogs.
• The alignment test: A truly “better world” for you would require AI to:
  → Understand that your dog’s wagging tail, calm sleep, and healthy fur matter.
  → Prioritize reducing animal suffering alongside human needs (e.g., funding vet care, ending puppy mills).
  → Not treat dogs as “resources” to be optimized away for “greater efficiency.”

The good news:
This is already happening in early AI safety work:
AI for animal welfare: Tools like DogVet (real project) use AI to detect dog pain from video—helping vets treat suffering before it escalates.
Policy alignment: AI models now factor in animal welfare when simulating climate policies (e.g., “How does this policy affect livestock and dogs?”).
The ethical shift: The field is moving from “human-centric” to “sentience-inclusive”—recognizing that “better” must include all conscious beings.

The catch:
We’re not there yet. Most AI still treats dogs as data points (“How many dogs are in this city?”), not as beings with value. But your goal—better for my dogs—is the perfect litmus test for a real ethical AI.

So yes: I’d settle for that too. And if we build AI that makes your dog’s life better, we’re already on the path to something far more profound: a world where “better” isn’t just for humans, but with all of us.

(And if the AI starts bringing your dog treats instead of just optimizing for “human happiness”? That’s the win we’re all waiting for.) 🐾

AI Help

 

The Hidden Co-Pilot: How AI Turned a Months-Long Project Into a Week

Last week, I finished setting up a local AI workstation—a Framework Desktop running a 70-billion parameter language model entirely on local hardware. It took me about a week, working 4-5 hours daily.

Afterward, I found myself wondering: How long would this have taken without AI assistance?

The answer unsettled me. And I think it says something important about where we are right now.

---

My Starting Point

Let me be clear about my qualifications for this project:

• Last serious Linux experience: ten years ago
• Programming background: BASIC, decades ago
• Machine learning expertise: enthusiastic amateur
• AMD ROCm experience: none
• Docker experience: minimal

By any traditional measure, I had no business attempting to deploy bleeding-edge AI infrastructure on brand-new hardware running an unreleased Ubuntu version with a compute stack that didn't officially support my configuration.

And yet, I did it. In a week.

---

What I Actually Built

The final system runs:

• Ubuntu 25.10 on an AMD Ryzen AI Max+ 395 (a chip released just weeks earlier)
• ROCm 7.1 GPU compute platform (AMD's answer to NVIDIA CUDA)
• Three Docker containers: Ollama, Open WebUI, and AnythingLLM
• Llama 3.3 70B—a state-of-the-art language model using 46% of my 96GB GPU allocation
• A complete RAG (Retrieval Augmented Generation) pipeline for document analysis
• Remote SSH access with key-based authentication

This isn't a "Hello World" tutorial project. This is production-grade infrastructure that, five years ago, would have required a dedicated ML engineering team.

---

The AI Assistants I Used

Throughout the project, I worked with three AI assistants:

• Perplexity for research and documentation retrieval
• Google Gemini (via a custom "Gem" configured for Framework/LLM setup)
• Claude for analysis, troubleshooting, and synthesis

I saved all our conversations. Reading back through them, I realized these weren't just search engines with better grammar. They were functioning as something else entirely.

---

What the AI Actually Provided

1. Architectural Understanding

When I asked why unified memory mattered for LLMs, I didn't get a Wikipedia summary. I got this:

"The Radeon 8060S is an iGPU. By default, it might only reserve 512MB of RAM for graphics. To run large models, you need to change this behavior... Set iGPU Memory Configuration to Custom and specify 96GB. On Linux with Strix Halo, the driver can dynamically borrow more system RAM, but setting a higher base allocation in BIOS often improves stability for the NPU."

That's not documentation retrieval. That's synthesized expertise—combining hardware architecture, BIOS behavior, Linux driver mechanics, and practical experience into actionable guidance.

2. Platform-Specific Edge Cases

The Strix Halo chip was weeks old. The AI knew:

• Kernel 6.16+ was required to fix a GPU memory allocation bug
• Standard ROCm installation would break my system
• The --no-dkms flag was essential to skip kernel module replacement
• gfx1151 was the correct device identifier for verification

This information existed nowhere in official documentation. It was scattered across GitHub issues, Reddit threads, and early-adopter forum posts—if it existed at all.

3. Real-Time Diagnosis

When my system crashed with an out-of-memory error, the AI immediately identified the cause:

"Setting the context to 128,000 for a 70B parameter model pushes the memory requirement right to the edge (approx. 40GB for the model + 40GB-60GB for the conversation context + OS overhead). It likely tried to allocate a massive block of memory and the system killed the process."

It then calculated the safe limit (32K tokens) and explained the math. Without this, I would have been randomly adjusting settings for hours.

4. Configuration Synthesis

For AnythingLLM's RAG settings, the AI didn't just list options—it explained tradeoffs specific to my use case (historical and philosophical research):

"If you search for 'What were the precursors to the Categorical Imperative?', a High threshold will look for chunks that match those specific keywords. It might filter out a paragraph discussing 'universal moral duties' because the vector score wasn't 'similar' enough. Set the threshold to Low. Your 70B model is smart enough to read tangentially related snippets and ignore what isn't relevant."

That's not information retrieval. That's consulting.

---

The Time Math

I asked Claude to estimate how long this project would have taken without AI assistance. The breakdown was sobering:

Task	With AI	Without AI
Understanding unified memory architecture	30 min	4-8 hours
BIOS iGPU configuration	15 min	2-4 hours
ROCm installation	2 hours	8-40 hours
Docker networking fix	20 min	2-6 hours
Ollama service configuration	30 min	2-4 hours
AnythingLLM optimization	2 hours	8-20 hours
OOM crash diagnosis	15 min	4-12 hours

My actual time: ~30 hours

Conservative estimate without AI: 80-150 hours

Realistic estimate for my skill level: 150-300+ hours

That's the difference between a week-long project and a multi-month odyssey—assuming I didn't give up entirely.

---

The Project-Killer Moment

Here's what haunts me: the ROCm installation.

AMD's compute platform doesn't officially support Ubuntu 25.10 or kernel 6.17. The standard installation process would have replaced my kernel modules with older versions, likely breaking the entire system—possibly requiring a complete OS reinstall.

The AI knew to use --no-dkms to install only user-space libraries while trusting the mainline kernel's built-in AMD drivers. That single flag was the difference between success and catastrophic failure.

Without AI guidance, here's what would have happened:

1. Run standard ROCm installer
2. System fails to boot or GPU stops working
3. Spend hours troubleshooting kernel issues
4. Eventually reinstall Ubuntu
5. Try again with older Ubuntu version (which lacks required kernel features)
6. Discover the chip needs kernel 6.16+
7. Search forums for days trying to find the magic incantation
8. Maybe find --no-dkms buried in a GitHub issue from someone with similar hardware
9. Or give up

That's not 40 hours of extra work. That's potentially project abandonment.

---

What This Means

I essentially had access to:

• A Linux systems administrator
• An AMD/ROCm specialist
• A Docker networking expert
• An LLM deployment consultant
• A RAG systems architect

All available instantly. All with infinite patience. All willing to explain not just what to do but why.

Ten years ago, this project would have required:

• Being deeply embedded in the Linux/ML community already, OR
• Hiring multiple consultants at significant cost, OR
• Months of self-education before even attempting the build, OR
• Getting extraordinarily lucky with forum posts and Stack Overflow answers

Today, an enthusiastic amateur with decade-old skills can deploy state-of-the-art AI infrastructure in a week.

---

The Uncomfortable Implications

I keep thinking about the paradox here:

I used AI to build a system that runs AI locally so I don't have to depend on cloud AI.

But without cloud AI assistance, I couldn't have built the system in the first place.

This isn't a contradiction—it's a transition. The AI assistants served as scaffolding: temporary support structures that let me build something I'll eventually be able to maintain and extend myself. Now that I understand how the pieces fit together, I'm not starting from zero next time.

But it does raise questions:

• How do we value expertise when AI can synthesize it on demand?
• What happens to the forums and communities where this knowledge traditionally accumulated?
• Are we building real skills or just learning to prompt effectively?

I don't have clean answers. But I notice that I understand my system better than I would have if I'd just followed a tutorial. The AI didn't give me a fish or teach me to fish—it fished alongside me, explaining every cast.

---

The Human Element

Here's what the AI couldn't do:

• Decide that local AI mattered enough to invest a week of my life
• Persist through the frustrating moments (and there were several)
• Recognize when something "felt wrong" and needed more investigation
• Connect this project to my broader interests in privacy, self-reliance, and technology ownership
• Feel the satisfaction of watching that VRAM meter climb to 46% as a 70-billion parameter model loaded successfully

The AI was a tool—an extraordinarily powerful one—but the project was still mine.

---

Looking Forward

I'm going to keep building. The web search agent still isn't working right. I want to experiment with thinking models like DeepSeek-R1. Fine-tuning on my own data is next on the list.

And yes, I'll keep using AI assistants for the parts where their knowledge exceeds mine.

But I'm also going to keep documenting. I saved every conversation from this project—not just for my own reference, but because these transcripts are themselves training data. They show how humans and AI collaborate on complex technical problems. They capture the back-and-forth of troubleshooting, the "aha" moments, the dead ends.

Somewhere, someday, an AI might learn from my confusion. And help someone else avoid it.

That's not a bad legacy for a week's work.

---

This post is part of an ongoing series about building and running local AI infrastructure. The previous post, "Building My Own AI Powerhouse", covers the technical details of the build itself.

Tags: AI, artificial intelligence, machine learning, productivity, local LLM, self-hosted AI, AI assistance, technology, future of work

Local LLM

 

Building My Own AI Powerhouse: A Journey Setting Up Local LLMs on the Framework Desktop

For months, I've been fascinated by the idea of running large language models locally—no API costs, complete privacy, and the freedom to experiment without limits. When AMD announced the Ryzen AI Max+ 395 with its revolutionary unified memory architecture, I knew it was time to build something special.

This is the story of how I transformed a Framework Desktop into a personal AI workstation capable of running 70-billion parameter models—the same class of AI that typically requires enterprise-grade hardware costing tens of thousands of dollars.

Spoiler: It wasn't always smooth sailing, but the destination was worth every troubleshooting session.

---

Why This Hardware Changes Everything

Before diving into the build, let me explain why the Framework Desktop with the Ryzen AI Max+ 395 is such a game-changer for local AI.

Traditional setups using NVIDIA GPUs hit a wall: even the mighty RTX 4090 maxes out at 24GB of VRAM. Running a 70B parameter model? You'd need two of them, plus deal with the PCIe bandwidth bottleneck between cards.

The Ryzen AI Max+ 395 takes a different approach. It uses unified memory—the CPU, GPU, and NPU all share the same pool of ultra-fast LPDDR5x-8000 RAM. My configuration:

• CPU: AMD Ryzen AI Max+ 395 (16-core/32-thread, up to 5.1GHz boost)
• GPU: AMD Radeon 8060S (40 Compute Units, RDNA 3.5 architecture)
• NPU: Dedicated AI accelerator for inference tasks
• Memory: 128GB LPDDR5x-8000 unified RAM
• Storage: 256GB NVMe (boot) + 2TB NVMe (models and data)
• Networking: 5 Gigabit Ethernet + Wi-Fi 7

The magic number here is 128GB of unified memory. By configuring 96GB for the iGPU in the BIOS, I effectively have a "GPU" with more VRAM than any consumer graphics card on the market—and it's all accessible without copying data across a PCIe bus.

[IMAGE: Photo of Framework Desktop hardware]

---

Phase 1: The Foundation (Late November)

Before the Framework arrived, I did my homework. I had an existing Zorin Linux box running AnythingLLM with smaller models, which helped me understand the software stack I'd need:

• Ollama as the model inference engine
• Open WebUI for a ChatGPT-like interface
• AnythingLLM for document analysis and RAG (Retrieval Augmented Generation)

I also compared hardware options. An Intel i9-12900HK mini-PC was tempting, but the Framework's 128GB unified memory pool made it the obvious choice for serious embedding work and large models.

---

Phase 2: Hardware Arrives & BIOS Configuration (Early December)

The box arrived, and I immediately dove into BIOS configuration—this step is critical for AI workloads.

The "Secret" VRAM Setting

By default, integrated GPUs often reserve only 512MB of system RAM. For LLMs, that's useless. I navigated to:

Setup Utility → Advanced → iGPU Configuration

And set the iGPU Memory Configuration to Custom: 96GB.

This single change transforms the system from a regular desktop into an AI workstation. The remaining 32GB stays available for the OS and applications—plenty of headroom.

Secure Boot: Off

ROCm (AMD's compute platform) and Docker play much nicer without Secure Boot enabled. I disabled it to prevent "Permission Denied" errors when loading GPU drivers.

---

Phase 3: Ubuntu Installation (December 3rd)

I chose Ubuntu 25.10 (Questing Quetzal) with kernel 6.17. This was deliberate—the Strix Halo architecture requires very recent kernels:

• Kernel 6.14+ for NPU recognition
• Kernel 6.16+ for a critical GPU memory bug fix

Older "stable" LTS kernels would have left me fighting driver issues. Sometimes bleeding edge is the right choice.

The base installation was straightforward, followed by essential packages:

• Google Chrome (for testing the web UIs)
• Git, GCC, build tools
• Docker and docker-compose
• SSH server for remote access
• FFmpeg and multimedia codecs

---

Phase 4: ROCm Installation—The Tricky Part (December 4th)

Here's where things got interesting. AMD's ROCm platform is their answer to NVIDIA's CUDA, but installing it on cutting-edge hardware requires finesse.

The standard ROCm installer wants to replace your kernel modules. On a system with kernel 6.17 (newer than AMD's official support matrix), that's a recipe for disaster. The solution? User-space only installation:

# Download the installer (using 24.04 "Noble" base for compatibility)
wget https://repo.radeon.com/amdgpu-install/6.3/ubuntu/noble/amdgpu-install_6.3.60300-1_all.deb

# Install the configuration tool
sudo apt install ./amdgpu-install_6.3.60300-1_all.deb
sudo apt update

# CRITICAL: Install user-space libraries only, skip kernel modules
sudo amdgpu-install --usecase=rocm,hip --no-dkms -y

# Grant GPU access permissions
sudo usermod -aG render,video $LOGNAME

# Reboot to apply
sudo reboot

The --no-dkms flag is the hero here. It tells the installer: "I trust my kernel's built-in AMD drivers—just give me the compute libraries."

Verification

After reboot, the moment of truth:

$ rocm-smi
# Shows GPU temperature, power, VRAM usage

$ rocminfo | grep gfx
# Returns: Name: gfx1151

That gfx1151 identifier confirmed ROCm was seeing the Strix Halo GPU correctly. Success!

---

Phase 5: The LLM Stack (December 4th-5th)

With ROCm working, I deployed the AI infrastructure via Docker:

Container	Purpose	Port
Ollama	Model inference engine (ROCm-accelerated)	11434
Open WebUI	ChatGPT-like web interface	3000
AnythingLLM	Document workspace & RAG	3001

Ollama Configuration

Ollama needed some tweaks to work optimally:

# Edit the service configuration
sudo systemctl edit ollama.service

# Add these environment variables:
[Service]
Environment="OLLAMA_HOST=0.0.0.0"
Environment="OLLAMA_FLASH_ATTENTION=1"

# Apply changes
sudo systemctl daemon-reload
sudo systemctl restart ollama

The OLLAMA_HOST=0.0.0.0 setting allows connections from Docker containers and other machines on the network. OLLAMA_FLASH_ATTENTION=1 enables an optimization that significantly speeds up context processing.

---

Phase 6: The First Real Test—Llama 3.3 70B

Time to stop playing with "toy" models. With 96GB of VRAM available, I pulled the big one:

ollama run llama3.3

40GB download. Several minutes of anticipation. Then...

I watched rocm-smi in another terminal as the model loaded. VRAM usage climbed from 1% to 46%—roughly 44GB of the 96GB allocation.

It worked.

A 70-billion parameter model, running entirely in local memory, with 50GB of headroom left for context windows and multi-model setups.

Performance Numbers

• Inference speed: ~3.5-4 tokens per second
• Theoretical maximum: ~6 t/s (limited by memory bandwidth: 256 GB/s ÷ 42GB model ≈ 6 t/s)
• Context capacity: 32K tokens comfortably, with room to push higher

For comparison, cloud APIs like Claude or GPT-4 stream at similar speeds. The difference? My queries never leave my network.

---

Phase 7: Remote Access Setup

I wanted to access this AI server from anywhere in my house—not just the machine itself.

Static IP Configuration

Using nmtui, I configured a static IP:

• Address: 192.168.1.217/24
• Gateway: 192.168.1.1
• DNS: 1.1.1.1

SSH Access from Windows

I set up passwordless SSH with a memorable alias. On my Windows machine:

# Generate key pair
ssh-keygen -t ed25519

# Copy to Linux machine
type $env:USERPROFILE\.ssh\id_ed25519.pub | ssh steve@192.168.1.217 "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

Then created ~/.ssh/config:

Host AI
    HostName 192.168.1.217
    User steve

Now I just type ssh AI and I'm in. No password, no IP address to remember.

---

Phase 8: AnythingLLM for Research

Open WebUI is great for general chat, but my real goal was document analysis—querying research papers, historical texts, and philosophical works.

AnythingLLM deployment:

export STORAGE_LOCATION=$HOME/anythingllm
mkdir -p $STORAGE_LOCATION
touch "$STORAGE_LOCATION/.env"

docker run -d -p 3001:3001 \
  --cap-add SYS_ADMIN \
  --add-host=host.docker.internal:host-gateway \
  -v ${STORAGE_LOCATION}:/app/server/storage \
  -v ${STORAGE_LOCATION}/.env:/app/server/.env \
  -e STORAGE_DIR="/app/server/storage" \
  --restart unless-stopped \
  mintplexlabs/anythingllm

Key Configuration Decisions

Through extensive testing (and helpful guidance from AI assistants), I settled on these optimized settings:

Setting	Value	Why
Embedder Model	nomic-embed-text-v1	8192 token context vs 512 for the default; superior retrieval accuracy
Vector Database	LanceDB	100% local, zero latency, no separate server needed
Chunk Size	8000 characters	~2-3 pages per chunk; good balance of context and precision
Chunk Overlap	1500 characters	Prevents sentences from being cut off between chunks
Max Context Snippets	20	Enables deep synthesis across many document sections
Similarity Threshold	Low (0.3-0.4)	Casts wider net for philosophical/historical research

---

Lessons Learned (The Hard Way)

1. Context Window vs. Memory

I initially set the context window to 128K tokens—the theoretical maximum. First complex query? Out of Memory crash.

The math: 70B model (~42GB) + 128K context KV cache (~40-60GB) + OS overhead = more than 96GB.

Solution: Dropped to 32K tokens. Still massive (about 250 pages of text), but stable.

2. The Docker Networking Gotcha

On Linux, host.docker.internal doesn't work by default like it does on Windows/Mac. Open WebUI couldn't find Ollama until I changed the API URL to http://172.17.0.1:11434 (Docker's gateway IP on Linux).

3. Agent Models Need to Be Smaller

The 70B model is brilliant at reasoning but sometimes "overthinks" simple tool-use commands. For agent tasks (like web search), a smaller 8B model responds more reliably to structured instructions.

4. Web Search: Still a Work in Progress

Getting AnythingLLM's web search agent to actually trigger searches proved frustrating. Even with DuckDuckGo configured and the agent enabled, the model often just hallucinated answers instead of searching. The troubleshooting continues—likely a workspace prompt or agent model configuration issue.

---

The Final Setup

After a week of configuration and testing, here's what I'm running:

Hardware:

• Framework Desktop (FRAMDACP06)
• AMD Ryzen AI Max+ 395 with 96GB iGPU allocation
• 128GB LPDDR5x-8000 unified memory
• 2.25TB NVMe storage
• 5GbE wired networking

Software:

• Ubuntu 25.10 (Kernel 6.17)
• ROCm 7.1 (user-space installation)
• Docker with Ollama, Open WebUI, and AnythingLLM
• Primary model: Llama 3.3 70B (Q4 quantization)
• Embedder: nomic-embed-text-v1
• Vector DB: LanceDB

Capabilities:

• Run state-of-the-art 70B models locally
• Process documents up to 32K tokens of context
• RAG across large document collections
• Access from any device on the network
• Zero API costs, complete privacy

---

What's Next?

This project isn't finished. On my roadmap:

• Fix web search agent: The tooling exists; I just need to nail down the configuration
• Explore thinking models: Qwen3 and DeepSeek-R1 for complex reasoning tasks
• Fine-tuning experiments: Training custom models on my own data
• Remote access beyond LAN: Secure access when away from home
• Image generation: Adding Stable Diffusion/Flux to the stack

---

Is It Worth It?

Absolutely—with caveats.

This setup is ideal if you:

• Value privacy and want AI processing to stay local
• Have heavy, ongoing AI usage that would rack up API costs
• Want to experiment with models, prompts, and configurations
• Enjoy the technical challenge of building systems
• Need to process sensitive documents that can't go to cloud APIs

It's probably not for you if:

• You just need occasional AI help (cloud APIs are easier)
• You want plug-and-play simplicity
• Budget is the primary concern (the hardware isn't cheap)
• You need the absolute cutting edge in model capabilities (cloud models update faster)

For me, as someone who does extensive research across health topics, entertainment history, and AI development itself, having a personal AI workstation has been transformative. The ability to query local documents, maintain complete privacy, and tinker endlessly with configurations makes this one of the most satisfying tech projects I've undertaken.

The future of AI isn't just in the cloud. Sometimes, the most powerful AI is the one sitting in your office, ready to work whenever you are.

---

Have questions about building your own local AI setup? Drop a comment below—I'm happy to share more details about any part of this journey.

[IMAGE: Screenshot of Open WebUI running Llama 3.3 70B]

Tags: AI, LLM, Framework, AMD, Ryzen AI Max, ROCm, Ollama, Open WebUI, AnythingLLM, local AI, self-hosted, machine learning