{"slug": "build-a-serverless-search-engine-with-dynamodb-vector-search-and-aws-cdk", "title": "Build a Serverless Search Engine with DynamoDB Vector Search and AWS CDK", "summary": "A developer built a serverless semantic search engine that combines DynamoDB vector search, Amazon Bedrock embeddings, AWS Lambda, and API Gateway, with infrastructure provisioned through AWS CDK. The project uses Titan Text Embeddings V2 and cosine-distance approximate nearest neighbor indexes to match queries by meaning rather than prefix-based lexical matching, addressing a limitation the developer encountered with help-article search at work. The writeup notes DynamoDB vector search became generally available on August 5, 2026.", "body_md": "Some time ago at my work, we had to implement a small search engine for help articles. The search was very simple: it used lexical, prefix-based matching. This was a problem because many words did not match.\n\nThat is exactly the kind of problem I wanted to solve with this project. Instead of relying solely on the article name or prefix matches, I wanted the search engine to understand the intent behind the query without sacrificing the speed and predictability of traditional search.\n\nIt was in this context that AWS announced support for vector indexes in DynamoDB, which was also the service where we were already storing the article data. The result is a small serverless search engine built with **DynamoDB vector search, Amazon Bedrock, AWS Lambda**, and **API Gateway**, with all the infrastructure provisioned using **AWS CDK**.\n\nThe main flow works as follows:\n\nYou do not need prior machine learning experience to follow this tutorial. Familiarity with TypeScript and basic AWS concepts should be enough.\n\nThis article reflects the project and AWS services available as of September 2026. DynamoDB vector search [became generally available](https://aws.amazon.com/about-aws/whats-new/2026/08/amazon-dynamodb-vector-search/) on August 5, 2026. Before using this design in production, review the official AWS documentation.\n\nAn embedding is essentially an array of numbers that represents the meaning of a piece of content.\n\nFor example, an embedding model can transform:\n\n```\nwaterproof shoes for mountain paths\n```\n\ninto something like:\n\n```\n[0.018, -0.041, 0.092, ... 509 more numbers]\n```\n\nThe important property is that texts with similar meanings tend to produce vectors that are close to one another.\n\nIn a product search engine, this allows us to build a relatively simple workflow:\n\nDynamoDB performs this final step using an **approximate nearest neighbor (ANN)** index. Its `SearchVectors` API returns the closest items according to the distance function configured for the index.\n\nFor this implementation, I chose cosine distance. With this metric, `0` represents identical vectors, and lower values indicate greater similarity.\n\nOne important detail is to maintain consistency across all embeddings. Both products and queries must use **the same model, number of dimensions, and normalization settings**.\n\nIn this project, I use:\n\nTitan Text Embeddings V2 supports 256, 512, and 1,024 dimensions, according to the [Amazon Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html).\n\nAt my work, we use help article titles, but for this tutorial we will use products.\n\nThe application has two separate flows: indexing and querying.\n\nThe indexing flow prepares products before they can be searched. The query flow processes searches coming from the frontend.\n\nAPI Gateway receives the requests, a Lambda function runs the search logic, and DynamoDB uses on-demand capacity.\n\nThe vector index projects the product attributes needed in the search response. This means a `SearchVectors` result already contains enough information to respond without performing another read.\n\nClone or open the [repository](https://github.com/ashelenlanube/serverless-search-engine-with-dynamo-vectors) and make sure you have:\n\n`bedrock:InvokeModel`.\nIn new Amazon Bedrock accounts, model access is usually enabled by default, although availability may still vary by Region. You can confirm Titan V2 availability in the [Bedrock model catalog and Region documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/models-region-compatibility.html).\n\nInstall the dependencies and run the local checks:\n\n```\nnpm install\nnpm run build\nnpm test\nnpm run lint\nnpm run format:check\n```\n\nIf this is the first CDK application you are deploying in that account and Region, initialize the environment once:\n\n```\nnpm run cdk -- bootstrap\n```\n\nThe central resource is an on-demand DynamoDB table whose partition key is the product `id`.\n\n``` js\nconst table = new dynamodb.Table(this, 'ProductsTable', {\n  partitionKey: { name: 'id', type: dynamodb.AttributeType.STRING },\n  billingMode: dynamodb.BillingMode.PAY_PER_REQUEST,\n  encryption: dynamodb.TableEncryption.AWS_MANAGED,\n  pointInTimeRecoverySpecification: { pointInTimeRecoveryEnabled: true },\n});\n```\n\nThe complete definition is available in [`infra/lib/search-engine-stack.ts`](https://github.com/ashelenlanube/serverless-search-engine-with-dynamo-vectors/blob/main/infra/lib/search-engine-stack.ts). Each item contains both the application’s regular data and its embedding:\n\n```\n{\n  \"id\": \"7381bede-ffcf-4efb-8d78-3db7168d2408\",\n  \"name\": \"Summit Trail Shoes\",\n  \"normalizedName\": \"summit trail shoes\",\n  \"nameInitial\": \"s\",\n  \"description\": \"Grippy trail shoes with a rock plate for wet mountain paths.\",\n  \"category\": \"footwear\",\n  \"tags\": [\"hiking\", \"trail\", \"waterproof\"],\n  \"score\": 1,\n  \"embeddingModel\": \"amazon.titan-embed-text-v2:0\",\n  \"embeddingDimensions\": 512,\n  \"embedding\": [0.018, -0.041, 0.092]\n}\n```\n\nThe actual `embedding` field contains 512 numbers.\n\nI also store `embeddingModel` and `embeddingDimensions`. These fields are not required to run the search, but keeping that metadata alongside the vector simplifies future model or dimensionality migrations.\n\nThe table also contains two additional GSIs:\n\n`AutocompleteIndex` enables prefix searches using the product’s normalized name.`PopularityIndex` sorts products by their number of clicks.\nNeither of these indexes is required for vector search. I added them because I did not want to rely solely on semantic similarity to rank the results.\n\nI placed the vector index creation logic in a reusable construct called `DynamoDbVectorIndex`:\n\n```\nnew DynamoDbVectorIndex(this, 'ProductEmbeddingVectorIndex', {\n  table,\n  indexName: 'ProductEmbeddingIndex',\n  vectorAttribute: 'embedding',\n  dimensions: 512,\n  distanceFunction: 'COSINE',\n  projectedAttributes: [\n    'name',\n    'normalizedName',\n    'description',\n    'category',\n    'tags',\n    'score',\n  ],\n});\n```\n\nThe number of dimensions must match the vectors generated by Titan. I chose cosine distance because the embeddings are normalized and I am more interested in comparing the direction of the vectors than their magnitude.\n\nCDK does not yet natively support adding a vector index, so I used an asynchronous CloudFormation [custom resource](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources.html).\n\nThe provider Lambda uses DynamoDB’s `UpdateTable` operation with `VectorIndexUpdates`:\n\n```\nawait client.send(\n  new UpdateTableCommand({\n    TableName: props.TableName,\n    VectorIndexUpdates: [\n      {\n        Create: {\n          IndexName: props.IndexName,\n          VectorAttribute: {\n            AttributeName: props.VectorAttribute,\n          },\n          Dimensions: props.Dimensions,\n          DistanceFunction: props.DistanceFunction,\n          Projection: {\n            ProjectionType: 'INCLUDE',\n            NonKeyAttributes: props.ProjectedAttributes,\n          },\n        },\n      },\n    ],\n  }),\n);\n```\n\nThis was one of the most interesting infrastructure details in the project.\n\nVector index creation is asynchronous. A successful response from `UpdateTable` **does not mean that the index is ready to use**. If CloudFormation considered the resource complete at that point, deployment could continue while the index was still being created.\n\nTo avoid a race condition, the custom resource provider calls `DescribeTable` every ten seconds and reports success only when the index reaches the `ACTIVE` state. It also handles deletion and replacement when immutable index properties change.\n\nThe complete implementation is available in:\n\nI find this pattern useful beyond this specific case: when CDK does not yet expose a feature directly, you can encapsulate the low-level AWS API and its associated logic within a reusable construct.\n\nCDK’s custom resource framework is very helpful here because it lets you separate the handler that starts the operation from the handler that checks when it has finished.\n\nAttribute projection also deserves attention. Returning the product fields directly from `SearchVectors` avoids an additional read, but projected attributes also consume vector index storage. That is why we project only the fields required by the response. AWS explains this trade-off in its [vector index storage documentation](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/VectorSearchStorage.html).\n\nAfter deployment, we have an empty table and vector index. Now we need to transform the product catalog into searchable data.\n\nThe loading script is located at [`scripts/seed.ts`](https://github.com/ashelenlanube/serverless-search-engine-with-dynamo-vectors/blob/main/scripts/seed.ts), while the sample catalog is in [`data/products.json`](https://github.com/ashelenlanube/serverless-search-engine-with-dynamo-vectors/blob/main/data/products.json).\n\nFor each product, I combine several fields into a single textual representation:\n\n```\nreturn [\n  `Name: ${product.name}`,\n  `Category: ${product.category}`,\n  `Description: ${product.description}`,\n  `Tags: ${product.tags.join(', ')}`,\n].join('\\n');\n```\n\nI could have generated embeddings from the product name alone, but that would discard much of the available context.\n\nIncluding the category, description, and tags gives the model more information about what each product actually represents.\n\nThe seed script then invokes Titan Embed V2:\n\n```\nnew InvokeModelCommand({\n  modelId: 'amazon.titan-embed-text-v2:0',\n  contentType: 'application/json',\n  accept: 'application/json',\n  body: new TextEncoder().encode(\n    JSON.stringify({\n      inputText,\n      dimensions: 512,\n      normalize: true,\n    }),\n  ),\n});\n```\n\nThe script also:\n\nBefore making calls to AWS, you can validate the catalog locally:\n\n```\nnpm run seed -- --dry-run\n```\n\nThat is usually the first command I run when I modify the sample data.\n\nOnce the catalog has been indexed, the browser can make requests such as:\n\n```\nGET /search?q=shoes%20for%20wet%20trails&limit=5\n```\n\nThe search Lambda first validates and normalizes the query.\n\nIt then starts two retrieval paths in parallel.\n\nThe first runs a DynamoDB `Query` against `AutocompleteIndex` to find products whose names begin with the text entered by the user.\n\nThe second generates an embedding for the query and uses `SearchVectors` to retrieve the 25 semantically closest candidates.\n\n``` js\nconst response = await dynamo.send(\n  new SearchVectorsCommand({\n    TableName: tableName,\n    IndexName: 'ProductEmbeddingIndex',\n    SearchVector: embedding.map((value) => ({\n      N: String(value),\n    })),\n    TopK: 25,\n  }),\n);\n```\n\nOne detail that is easy to misinterpret is the `Score` returned by the vector search.\n\nBecause the index uses **cosine distance**, lower values are better. This is not a traditional relevance score in which a higher number means a better result.\n\nFor ranking, I convert that distance into a normalized similarity value:\n\n```\nsemantic score = 1 - cosine distance / 2\n```\n\nThe result is clamped to the range between zero and one.\n\nFor example:\n\n``` php\ndistance = 0  -> semantic score = 1\ndistance = 2  -> semantic score = 0\n```\n\nI decided not to use vector similarity as the final ranking mechanism. Traditional lexical search is still very effective in certain cases.\n\nIf a user types `cloud run` and the catalog contains `Cloud Runner Shoes`, I would expect that product to appear near the top, even if another product happens to be semantically similar.\n\nThe Lambda therefore combines candidates by product ID and calculates the following weighted score:\n\n```\nranking score =\n    0.50 × lexical score\n  + 0.45 × semantic score\n  + 0.05 × popularity score\n```\n\nThe complete implementation is in [`packages/shared/src/ranking.ts`](https://github.com/ashelenlanube/serverless-search-engine-with-dynamo-vectors/blob/main/packages/shared/src/ranking.ts).\n\nThe weights slightly favor lexical matches. Semantic similarity retrieves results that a text search would not find, while popularity accounts for only 5%, mainly as a tiebreaker between candidates with similar scores.\n\nThese weights are not universal. They are simply a starting point for this example. In a real product search engine, I would tune them using real queries and expected results instead of defining them based solely on intuition.\n\nAnother decision was to run both retrieval paths with `Promise.allSettled`. This gives the endpoint useful fallback behavior. If Bedrock fails temporarily, the lexical search can still return results. The entire request fails only when both paths fail.\n\nFor a search endpoint, returning a slightly worse result is better than returning no result at all.\n\nThe search Lambda mainly needs two groups of permissions.\n\nThe first allows it to query DynamoDB:\n\n```\nfunctions.search.addToRolePolicy(\n  new iam.PolicyStatement({\n    actions: ['dynamodb:Query', 'dynamodb:SearchVectors'],\n    resources: [\n      table.tableArn,\n      `${table.tableArn}/index/AutocompleteIndex`,\n      `${table.tableArn}/index/ProductEmbeddingIndex`,\n    ],\n  }),\n);\n```\n\nThe second allows it to invoke only the embedding model used by the application:\n\n```\nfunctions.search.addToRolePolicy(\n  new iam.PolicyStatement({\n    actions: ['bedrock:InvokeModel'],\n    resources: [\n      `arn:${cdk.Aws.PARTITION}:bedrock:${cdk.Aws.REGION}::foundation-model/amazon.titan-embed-text-v2:0`,\n    ],\n  }),\n);\n```\n\nThe other functions receive more specific permissions.\n\nThe popular-products Lambda needs only `dynamodb:Query`, while the Lambda responsible for recording clicks needs only `dynamodb:UpdateItem`. This keeps each execution role limited to the actions required by that function instead of granting broad access across the application.\n\nFirst, generate the CloudFormation template locally:\n\n```\nnpm run cdk:synth\n```\n\nThen deploy the development stack:\n\n```\nnpm run cdk:deploy -- --context stage=dev\n```\n\nThe deployment saves its outputs to:\n\n```\ncdk-outputs.json\n```\n\nThe seed script uses that file to obtain the generated names of the DynamoDB table and vector index.\n\nLoad the catalog by running:\n\n```\nnpm run seed\n```\n\nTo start the React interface, copy the environment variables file:\n\n```\ncp apps/web/.env.example apps/web/.env.local\n```\n\nSet:\n\n```\nVITE_API_BASE_URL\n```\n\nto the `ApiUrl` value generated in `cdk-outputs.json`.\n\nThen start Vite:\n\n```\nnpm run dev:web\n```\n\nOpen `http://localhost:5173` in your browser, and you will see the application with the search interface.\n\nThe most interesting searches are not necessarily product names. Try queries that express an intent:\n\n```\nshoes for wet mountain trails\nquiet keyboard for programming\nsomething that keeps drinks cold\ngear for working out while traveling\n```\n\nIn those cases, it is much easier to see the difference between lexical and semantic retrieval.\n\n`AccessDeniedException` from Amazon Bedrock\nVerify that Titan Text Embeddings V2 is available in the selected Region and that your user has permission to invoke it.\n\nThe deployed Lambda receives the required permission through CDK, but the local credentials used by the seed script also need `bedrock:InvokeModel`.\n\nThe project’s deployment command generates `cdk-outputs.json`.\n\nRun:\n\n```\nnpm run cdk:deploy -- --context stage=dev\n```\n\nbefore running the seed.\n\nYou can also provide both resources explicitly:\n\n```\nnpm run seed -- \\\n  --table-name YOUR_TABLE \\\n  --vector-index-name ProductEmbeddingIndex \\\n  --region YOUR_REGION\n```\n\nThree values must always match:\n\n`dimensions`.\nThe example uses `512`.\n\nOne small piece of technical debt in the project is that this constant still appears separately in the infrastructure, seed, and runtime code.\n\nIf you change the number of dimensions, you will need to update all three places and rebuild both the index and the stored embeddings.\n\nThis is **normal**. Vector index creation is asynchronous, so the custom resource may spend several minutes waiting for DynamoDB to report that the index is `ACTIVE`.\n\nThe provider allows up to 30 minutes for this process. As an additional safety measure, the seed script checks the index status again before inserting vectors.\n\nEmbeddings understand semantic relationships in text, but they do not automatically know your application’s business rules.\n\nIf the search quality is poor, the first thing I would review is the text used to generate each embedding.\n\nFor example, using only the product name may not provide enough information. Adding descriptions, categories, and tags can make a significant difference.\n\nAfter that, I would test the system with real queries and tune the ranking weights using an evaluation set.\n\nFor a larger catalog, other signals—such as category filters, inventory availability, or business-specific rules—will probably become just as important as vector similarity.\n\nThis is probably the most important point if the search engine starts serving real users.\n\nCreate a fixed set of representative queries and define which products should appear in the top positions for each one.\n\nThen run the same evaluation every time you change:\n\nOtherwise, it is very easy to make the search “feel better” for one query while making several others worse.\n\nWhenever relevant product information changes, its embedding should be regenerated.\n\nModel migrations also require planning.\n\nThe `embeddingModel` and `embeddingDimensions` fields stored alongside the product can help identify which vectors need to be rebuilt or which ones can temporarily coexist during a migration.\n\nAWS currently documents a maximum of 4,096 dimensions, up to five vector indexes per table by default, and `TopK` values of up to 100.\n\nBecause these limits may change, use the [DynamoDB quotas documentation](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ServiceQuotas.html) as the source of truth.\n\nWhen you have finished experimenting, delete the development resources:\n\n```\nnpm run cdk:destroy -- --context stage=dev\n```\n\nThe stack uses a `destroy` removal policy and disables deletion protection for `dev` and `demo` environments.\n\nIn production, the default behavior retains the table and enables deletion protection.\n\nThe code for this tutorial is available in this [repository](https://github.com/ashelenlanube/serverless-search-engine-with-dynamo-vectors).", "url": "https://wpnews.pro/news/build-a-serverless-search-engine-with-dynamodb-vector-search-and-aws-cdk", "canonical_source": "https://dev.to/aws-builders/build-a-serverless-search-engine-with-dynamodb-vector-search-and-aws-cdk-4j4p", "published_at": "2026-09-16 02:00:35+00:00", "updated_at": "2026-09-16 02:07:03.067924+00:00", "lang": "en", "topics": ["ai-tools", "ai-infrastructure", "developer-tools", "natural-language-processing", "mlops"], "entities": ["AWS", "DynamoDB", "Amazon Bedrock", "AWS Lambda", "API Gateway", "AWS CDK", "Titan Text Embeddings V2"], "alternates": {"html": "https://wpnews.pro/news/build-a-serverless-search-engine-with-dynamodb-vector-search-and-aws-cdk", "markdown": "https://wpnews.pro/news/build-a-serverless-search-engine-with-dynamodb-vector-search-and-aws-cdk.md", "text": "https://wpnews.pro/news/build-a-serverless-search-engine-with-dynamodb-vector-search-and-aws-cdk.txt", "jsonld": "https://wpnews.pro/news/build-a-serverless-search-engine-with-dynamodb-vector-search-and-aws-cdk.jsonld"}}