From Tedious Search to AI Insight: How I Turned a 690-Page Blog into a Personal Research Assistant

In the vast landscape of the internet, some voices stand out as beacons of knowledge and experience. For many, the "Lympho Bob" blog is one such beacon—a detailed, decade-spanning chronicle of one person's journey with Follicular Lymphoma. It’s an invaluable resource, but like many older blogs, its native search bar can be tedious, making it difficult to find specific information within its extensive archives.

This was the exact challenge I faced. I wanted to deeply understand Bob's experience, but sifting through years of posts was daunting. I knew there had to be a better way. This led me on a technical journey to convert his entire blog into a single, searchable document, ultimately creating a 690-page PDF that I could use as a private, AI-powered knowledge base.

Here’s how it was done.

---

Step 1: Rescuing the Blog with HTTrack

The first step was to get the entire blog off the internet and onto my computer. For this, I used WinHTTrack Website Copier, a fantastic open-source tool that creates a complete offline copy of a website. By pointing HTTrack at Lympho Bob's blog URL, it meticulously crawled and downloaded every post, page, and image, leaving me with a local folder full of organized .html files—the raw material for the next stage.

Step 2: Transforming the Content with Pandoc & PowerShell

With the raw HTML files in hand, the goal was to consolidate them into a single, clean document. This is where the magic of Pandoc, a universal document converter, came into play. Operating from the PowerShell terminal, Pandoc became the engine for this transformation.

Instead of manually converting hundreds of files, I used PowerShell scripting to automate the entire process. The workflow looked like this:

1. Targeting the Files: I first wrote simple scripts to find all the downloaded .html files, even those buried in subfolders organized by year.
2. Automated Conversion: The scripts then instructed Pandoc to process all of these files. We explored converting them into various formats like .docx, .md, and finally, the .odt format that would lead to our PDF.
3. Combining and Organizing: The most crucial step was telling Pandoc to merge everything into one single document. We also used its built-in feature to automatically generate a clickable table of contents, which is essential for navigating a document of this size.

Through a bit of trial and error, and some script refinement, we created an automated process that could reliably turn a chaotic folder of HTML files into one polished, organized manuscript.

Step 3: Unleashing the Power of NotebookLM

This is where the project truly came to life. I took the final, 690-page converted document and uploaded it as a source into Google's NotebookLM, an AI-powered research and note-taking assistant.

Instantly, the entire blog was transformed. It was no longer a series of disconnected posts but a unified, intelligent knowledge base. The tedious search bar was gone. In its place, I could now:

• Ask direct questions: I could ask specific questions like, "What were Lympho Bob's experiences with [a specific treatment]?" and get a direct, summarized answer.
• Get cited sources: NotebookLM didn't just give me an answer; it cited the exact pages in the document where it found the information, allowing me to jump straight to the source material.
• Summarize and synthesize: I could ask the AI to summarize themes or track developments across years of posts.

The hours I spent on the conversion process saved me countless more in research. The personal experience and deep knowledge shared by Lympho Bob were now fully accessible to me, on my terms.

---

The Takeaway

This journey from a hard-to-search blog to a personal AI research assistant shows how a few powerful, free tools can unlock the value hidden in online content. By combining a web copier (HTTrack), a document converter (Pandoc), and an AI interface (NotebookLM), we can preserve valuable knowledge and, more importantly, interact with it in entirely new ways. It’s a powerful reminder that sometimes, the best way to find the answer you’re looking for is to build a better way to ask the question.