findintermediate9 min readAug 29, 2026part 1 of 6

Set up CLIP and index a photo folder offline

Install CLIP, download the weights once, cut the network, and turn a folder of photos into a searchable index.

Before you start

Python3.10 or newer
about600 MB free for model weights
a folder of photos

By the end of this you will have two files, vecs.npy and paths.json, that together make every photo in a folder searchable by description. The tool will run with your network cable unplugged.

Set up an isolated environment#

mkdir imgsearch
cd imgsearch
python -m venv .venv
.venv\Scripts\activate

On macOS or Linux the last line is source .venv/bin/activate.

Install the packages#

python -m pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
python -m pip install open_clip_torch pillow numpy scikit-learn flask

The first line pulls the CPU-only torch build, around 200 MB. If you have an Nvidia card, drop the --index-url argument and you get the CUDA build instead, roughly 2.5 GB but far faster at indexing.

Download the model weights#

python -c "import open_clip; open_clip.create_model_and_transforms('ViT-B-32', pretrained='laion2b_s34b_b79k')"

That pulls roughly 600 MB into ~/.cache/huggingface, or C:\Users\<you>\.cache\huggingface on Windows. This is the only step in the entire series that requires an internet connection.

Cut the network#

Set HF_HUB_OFFLINE=1 in your environment, or put this at the very top of your scripts:

import os
os.environ["HF_HUB_OFFLINE"] = "1"

import open_clip   # must come after

Why this works#

huggingface_hub reads that variable at import time. If you set it below the import, it silently does nothing and you will believe you are offline when you are not. This is the single most common way people get this wrong.

To verify, disconnect from the network and run:

python -c "import os; os.environ['HF_HUB_OFFLINE']='1'; import open_clip; open_clip.create_model_and_transforms('ViT-B-32', pretrained='laion2b_s34b_b79k'); print('loaded offline')"

If it prints, you are fully local. If it complains about a missing file, the download did not finish and you need to redo the previous step with a connection.

Be aware the variable makes hub lookups fail loudly instead of quietly reaching for the network. That is the point, but it means a typo in a model name gives you a cache-miss error rather than a clear "no such model".

Write the indexer#

# index.py
import os
os.environ.setdefault("HF_HUB_OFFLINE", "1")

import json, sys, time
from pathlib import Path

import numpy as np
import torch
import open_clip
from PIL import Image, ImageFile

ImageFile.LOAD_TRUNCATED_IMAGES = True

VECS  = Path("vecs.npy")
PATHS = Path("paths.json")
EXTS  = {".jpg", ".jpeg", ".png", ".webp", ".bmp", ".gif", ".tif", ".tiff"}

def main(folder, batch=32):
    root = Path(folder)
    if not root.is_dir():
        sys.exit(f"not a folder: {root}")

    # resume support: only embed what we have not seen
    if VECS.exists() and PATHS.exists():
        old_vecs  = np.load(VECS)
        old_paths = json.loads(PATHS.read_text())
    else:
        old_vecs, old_paths = None, []

    known = set(old_paths)
    todo = [p for p in sorted(root.rglob("*"))
            if p.suffix.lower() in EXTS and str(p) not in known]
    if not todo:
        print("nothing new to index")
        return
    print(f"{len(todo)} new images")

    device = "cuda" if torch.cuda.is_available() else "cpu"
    print(f"device: {device}")
    model, _, preproc = open_clip.create_model_and_transforms(
        "ViT-B-32", pretrained="laion2b_s34b_b79k")
    model.eval().to(device)

    new_vecs, new_paths, skipped = [], [], 0
    t0 = time.time()

    with torch.no_grad():
        for i in range(0, len(todo), batch):
            imgs, ok = [], []
            for p in todo[i:i + batch]:
                try:
                    imgs.append(preproc(Image.open(p).convert("RGB")))
                    ok.append(str(p))
                except Exception as e:
                    skipped += 1
                    print(f"\nskip {p.name}: {e}")
            if not imgs:
                continue

            v = model.encode_image(torch.stack(imgs).to(device))
            v = v / v.norm(dim=-1, keepdim=True)
            new_vecs.append(v.cpu().numpy().astype(np.float32))
            new_paths += ok

            done = len(new_paths) + skipped
            rate = done / max(time.time() - t0, 0.001)
            print(f"  {done}/{len(todo)}  {rate:.1f}/s", end="\r", flush=True)

    stacked = np.vstack(new_vecs)
    if old_vecs is not None:
        stacked = np.vstack([old_vecs, stacked])
        new_paths = old_paths + new_paths

    np.save(VECS, stacked)
    PATHS.write_text(json.dumps(new_paths))
    print(f"\nindexed {len(new_paths)} total, {skipped} skipped")

if __name__ == "__main__":
    main(sys.argv[1])

Run it:

python index.py "D:\pics"

Why this works#

Normalizing the vectors. The v / v.norm(...) line scales every embedding to unit length. After that, a dot product between any two of them is exactly their cosine similarity, so searching later is one matrix multiply with no extra math.

Batching. Feeding 32 images through at once is substantially faster than one at a time, because the matrix operations underneath are far more efficient at scale. Larger batches use more memory for diminishing returns.

torch.no_grad(). Gradients are for training. We are only running the model forward, so this tells torch to skip building the computation graph, which saves both memory and time.

Resume support. The known set means running the script again only embeds files it has not seen. Add fifty photos to a folder of twenty thousand and you wait for fifty, not twenty thousand.

LOAD_TRUNCATED_IMAGES = True. Real photo libraries contain slightly damaged files. Without this, one truncated JPEG stops the whole run.

Storing paths, not images. paths.json holds the location of each file. No image data is copied anywhere. The tradeoff is that moving or renaming the source folder breaks the index, since the paths no longer resolve.

What to expect while it runs#

On a Windows laptop with the CPU-only torch build and batch size 32, indexing 107 images ran at 8.3 images per second, finishing in about 0.2 minutes. Broadly, CPU indexing lands somewhere in the range of 10 to 20 images per second depending on your machine, and a GPU is roughly an order of magnitude faster than that. Only the 8.3 figure here is measured.

So a folder of 20,000 photos is a long lunch on a CPU or a coffee break on a GPU.

You will probably see warnings like these:

UserWarning: Truncated File Read
UserWarning: Image appears to be a malformed MPO file, it will be
interpreted as a base JPEG file

Both are harmless. The first is the truncation setting doing its job. The second comes from phone photos that embed a second frame, and PIL reading just the primary image is exactly what you want.

Check the result#

import numpy as np, json
v = np.load("vecs.npy")
p = json.loads(open("paths.json").read())
print(v.shape, len(p))          # (107, 512) 107
print(round(float(np.linalg.norm(v[0])), 4))   # 1.0

Two numbers to confirm. The shape should be one row per image with 512 columns, and the row count should match the path count. Any embedding's norm should be 1.0, proving the normalization worked.

Your vecs.npy comes to roughly 2 KB per image, so 20,000 photos is about 40 MB.

Where this stops being the right approach#

If your photo folder lives on OneDrive with files set to online-only, the paths resolve but reading each one triggers a download. Indexing works and is slow, and it pulls files back onto your disk. Either accept that or mark the folder as always available first.

If you need to index millions of images, a flat .npy file loaded entirely into memory stops being reasonable somewhere in the low millions. At that point move to a vector database with an approximate nearest neighbor index. For a personal photo library you will not get close.

Next: how the search actually compares a sentence to a photo, and the specific reason it plateaus.

More on finding things in your own files