Semantic Image Search

Features

Next.js App Router
React Server Components (RSCs), Suspense, and Server Actions
Vercel AI SDK for multimodal prompting, generating & embedding image metadata, and streaming images from Server to Client
Support for OpenAI (default), Gemini, Anthropic, Cohere, or custom AI chat models
shadcn/ui
- Styling with Tailwind CSS
- Radix UI for headless component primitives
Query caching with Vercel KV
Embeddings powered by Vercel Postgres, pgvector, and Drizzle ORM
File (image) storage with Vercel Blob

Model Providers

This template ships with OpenAI GPT-4o as the default. However, thanks to the Vercel AI SDK, you can switch LLM providers to Gemini, Anthropic, Cohere, and more with just a few lines of code.

Deploy Your Own

You can deploy your own version of the Semantic Image Search App to Vercel with one click:

Setup

Creating a KV Database Instance

Follow the steps outlined in the quick start guide provided by Vercel. This guide will assist you in creating and configuring your KV database instance on Vercel, enabling your application to interact with it.

Remember to update your environment variables (KV_URL, KV_REST_API_URL, KV_REST_API_TOKEN, KV_REST_API_READ_ONLY_TOKEN) in the .env file with the appropriate credentials provided during the KV database setup.

Creating a Postgres Database Instance

Follow the steps outlined in the quick start guide provided by Vercel. This guide will assist you in creating and configuring your Postgres database instance on Vercel, enabling your application to interact with it.

Once you have instantiated your Vercel Postgres instance, run the following code to enable pgvector:

CREATE EXTENSION vector;

Remember to update your environment variables (POSTGRES_URL, POSTGRES_PRISMA_URL, POSTGRES_URL_NON_POOLING, POSTGRES_USER, POSTGRES_HOST, POSTGRES_PASSWORD, POSTGRES_DATABASE) in the .env file with the appropriate credentials provided during the Postgres database setup.

Creating a Blob Instance

Follow the steps outlined in the quick start guide provided by Vercel. This guide will assist you in creating and configuring your Blob instance on Vercel, enabling your application to interact with it.

Remember to update your environment variable (BLOB_READ_WRITE_TOKEN) in the .env file with the appropriate credentials provided during the Blob setup.

Running locally

You will need to use the environment variables defined in .env.example to run Next.js Semantic Image Search. It's recommended you use Vercel Environment Variables for this, but a .env file is all that is necessary.

Note: You should not commit your .env file or it will expose secrets that will allow others to control access to your various OpenAI and authentication provider accounts.

Install Vercel CLI: npm i -g vercel
Link local instance with Vercel and GitHub accounts (creates .vercel directory): vercel link
Download your environment variables: vercel env pull

pnpm install

Add OpenAI API Key

Be sure to add your OpenAI API Key to your .env.

Database Setup

To push your schema changes to your Vercel Postgres database, run the following command.

pnpm run db:generate
pnpm run db:push

Prepare your Images (Indexing Step)

To get your application ready for Semantic search, you will have to complete three steps.

Upload Images to storage
Send Images to a Large Language Model to generate metadata (title, description)
Iterate over each image, embed the metadata, and then save to the database

Upload Images

Put the images you want to upload in the images-to-index directory (.jpg format) at the root of your application. Run the following command.

pnpm run upload

This script will upload the images to your Vercel Blob store. Depending on how many photos you are uploading, this step could take a while.

Generate Metadata

Run the following command.

pnpm run generate-metadata

This script will generate metadata for each of the images you uploaded in the previous step. Depending on how many photos you are uploading, this step could take a while.

Embed Metadata and Save to Database

Run the following command.

pnpm run embed-and-save

Depending on how many photos you are uploading, this step could take a while. This script will embed the descriptions generated in the previous step and save them to your Vercel Postgres instance.

Starting the Server

Run the following command

pnpm run dev

Your app template should now be running on localhost:3000.

Authors

This library is created by Vercel and Next.js team members, with contributions from:

Jared Palmer (@jaredpalmer) - Vercel
Shu Ding (@shuding_) - Vercel
shadcn (@shadcn) - Vercel
Lars Grammel (@lgrammel) - Vercel
Nico Albanese (@nicoalbanese10) - Vercel