{"slug": "building-a-semantic-search-api-in-go-with-meilisearch", "title": "Building a semantic search API in Go with Meilisearch", "summary": "This article provides a tutorial on building a semantic search API in Go using the Fiber web framework and Meilisearch, a typo-tolerant search engine. The architecture includes filter support, configurable typo tolerance, and a MySQL LIKE query fallback for resilience, and is used to power search across over 1,600 cybersecurity articles. The guide covers project setup, index configuration with searchable and filterable attributes, and document indexing.", "body_md": "Full-text search is one of those features that looks simple until you have to ship it. Typos fail silently. Category filters conflict with relevance ranking. The database LIKE query that worked at 10,000 rows grinds to a halt at 100,000. This tutorial walks through building a real search API in Go using Fiber and Meilisearch, complete with filter support, typo tolerance configuration, and a MySQL LIKE fallback for resilience.\n\nThis is roughly the architecture running search across 1,600+ cybersecurity articles at [AYI NEDJIMI Consultants](https://ayinedjimi-consultants.fr/articles).\n\n## Setup\n\n```\ngo mod init search-api\ngo get github.com/gofiber/fiber/v2\ngo get github.com/meilisearch/meilisearch-go\ngo get github.com/go-sql-driver/mysql\n```\n\nRun Meilisearch:\n\n```\ndocker run -d -p 7700:7700 \\\n  -e MEILI_MASTER_KEY=your_master_key \\\n  getmeili/meilisearch:latest\n```\n\n## Project structure\n\n```\nsearch-api/\n├── main.go\n├── config/\n│   └── config.go\n├── search/\n│   ├── meili.go\n│   └── fallback.go\n└── handlers/\n    └── search.go\n```\n\n## Data model\n\n```\n// search/meili.go\npackage search\n\n// Article is the document type stored in Meilisearch and MySQL.\ntype Article struct {\n    ID         string   `json:\"id\"`\n    Title      string   `json:\"title\"`\n    Slug       string   `json:\"slug\"`\n    Content    string   `json:\"content\"`    // plain text, stripped of HTML\n    Category   string   `json:\"category\"`   // news, guide, analyse, blog, checklist\n    Difficulty string   `json:\"difficulty\"` // beginner, intermediate, advanced\n    DocType    string   `json:\"doc_type\"`   // article, checklist, glossary\n    Tags       []string `json:\"tags\"`\n    PublishedAt int64   `json:\"published_at\"` // Unix timestamp for sort\n}\n\n// SearchResult wraps hits with metadata.\ntype SearchResult struct {\n    Hits             []Article `json:\"hits\"`\n    TotalHits        int64     `json:\"total_hits\"`\n    ProcessingTimeMs int64     `json:\"processing_time_ms\"`\n    Query            string    `json:\"query\"`\n    Source           string    `json:\"source\"` // \"meilisearch\" or \"mysql_fallback\"\n}\n```\n\n## Meilisearch client initialization\n\n```\n// search/meili.go (continued)\npackage search\n\nimport (\n    \"fmt\"\n    \"log\"\n\n    \"github.com/meilisearch/meilisearch-go\"\n)\n\nconst IndexName = \"articles\"\n\ntype MeiliSearcher struct {\n    client meilisearch.ServiceManager\n    index  meilisearch.IndexManager\n}\n\nfunc NewMeiliSearcher(host, apiKey string) (*MeiliSearcher, error) {\n    client := meilisearch.New(host, meilisearch.WithAPIKey(apiKey))\n\n    // Verify connectivity\n    if _, err := client.Health(); err != nil {\n        return nil, fmt.Errorf(\"meilisearch unreachable at %s: %w\", host, err)\n    }\n\n    s := &MeiliSearcher{client: client}\n    if err := s.ensureIndex(); err != nil {\n        return nil, err\n    }\n    return s, nil\n}\n\nfunc (s *MeiliSearcher) ensureIndex() error {\n    // Get or create index\n    idx, err := s.client.GetIndex(IndexName)\n    if err != nil {\n        task, err := s.client.CreateIndex(&meilisearch.IndexConfig{\n            Uid:        IndexName,\n            PrimaryKey: \"id\",\n        })\n        if err != nil {\n            return fmt.Errorf(\"create index: %w\", err)\n        }\n        s.client.WaitForTask(task.TaskUID, nil)\n        idx, err = s.client.GetIndex(IndexName)\n        if err != nil {\n            return fmt.Errorf(\"get index after create: %w\", err)\n        }\n    }\n    s.index = idx\n    return s.configureIndex()\n}\n\nfunc (s *MeiliSearcher) configureIndex() error {\n    task, err := s.index.UpdateSettings(&meilisearch.Settings{\n        SearchableAttributes: []string{\n            \"title\",      // highest weight\n            \"tags\",\n            \"content\",    // lowest weight\n        },\n        FilterableAttributes: []string{\n            \"category\",\n            \"difficulty\",\n            \"doc_type\",\n        },\n        SortableAttributes: []string{\n            \"published_at\",\n        },\n        RankingRules: []string{\n            \"words\",\n            \"typo\",\n            \"proximity\",\n            \"attribute\",   // respects SearchableAttributes order\n            \"sort\",\n            \"exactness\",\n        },\n        TypoTolerance: &meilisearch.TypoTolerance{\n            Enabled: func() *bool { b := true; return &b }(),\n            MinWordSizeForTypos: meilisearch.MinWordSizeForTypos{\n                OneTypo:  4,\n                TwoTypos: 8,\n            },\n        },\n        Pagination: &meilisearch.Pagination{\n            MaxTotalHits: 10000,\n        },\n    })\n    if err != nil {\n        return fmt.Errorf(\"update settings: %w\", err)\n    }\n    s.client.WaitForTask(task.TaskUID, nil)\n    log.Println(\"Meilisearch index configured.\")\n    return nil\n}\n```\n\n## Index sync function\n\nCall this on startup and hook it to your CRUD operations.\n\n```\n// search/meili.go (continued)\n\nfunc (s *MeiliSearcher) IndexDocuments(articles []Article) error {\n    if len(articles) == 0 {\n        return nil\n    }\n    task, err := s.index.AddDocuments(articles, \"id\")\n    if err != nil {\n        return fmt.Errorf(\"add documents: %w\", err)\n    }\n    s.client.WaitForTask(task.TaskUID, nil)\n    log.Printf(\"Indexed %d documents.\", len(articles))\n    return nil\n}\n\nfunc (s *MeiliSearcher) DeleteDocument(id string) error {\n    task, err := s.index.DeleteDocument(id)\n    if err != nil {\n        return fmt.Errorf(\"delete document %s: %w\", id, err)\n    }\n    s.client.WaitForTask(task.TaskUID, nil)\n    return nil\n}\n```\n\n## Search with filters\n\n```\n// search/meili.go (continued)\n\ntype SearchParams struct {\n    Query      string\n    Category   string\n    Difficulty string\n    DocType    string\n    Limit      int64\n    Offset     int64\n}\n\nfunc (s *MeiliSearcher) Search(params SearchParams) (*SearchResult, error) {\n    if params.Limit <= 0 {\n        params.Limit = 10\n    }\n    if params.Limit > 100 {\n        params.Limit = 100\n    }\n\n    // Build filter expression\n    var filters []string\n    if params.Category != \"\" {\n        filters = append(filters, fmt.Sprintf(\"category = %q\", params.Category))\n    }\n    if params.Difficulty != \"\" {\n        filters = append(filters, fmt.Sprintf(\"difficulty = %q\", params.Difficulty))\n    }\n    if params.DocType != \"\" {\n        filters = append(filters, fmt.Sprintf(\"doc_type = %q\", params.DocType))\n    }\n\n    filterStr := \"\"\n    for i, f := range filters {\n        if i == 0 {\n            filterStr = f\n        } else {\n            filterStr += \" AND \" + f\n        }\n    }\n\n    req := &meilisearch.SearchRequest{\n        Limit:  params.Limit,\n        Offset: params.Offset,\n        AttributesToRetrieve: []string{\n            \"id\", \"title\", \"slug\", \"category\",\n            \"difficulty\", \"doc_type\", \"tags\", \"published_at\",\n        },\n        // Don't return full content in search results\n    }\n    if filterStr != \"\" {\n        req.Filter = filterStr\n    }\n\n    resp, err := s.index.Search(params.Query, req)\n    if err != nil {\n        return nil, err\n    }\n\n    hits := make([]Article, 0, len(resp.Hits))\n    for _, h := range resp.Hits {\n        // Meilisearch returns hits as map[string]interface{}\n        b, _ := json.Marshal(h)\n        var a Article\n        if err := json.Unmarshal(b, &a); err == nil {\n            hits = append(hits, a)\n        }\n    }\n\n    return &SearchResult{\n        Hits:             hits,\n        TotalHits:        resp.TotalHits,\n        ProcessingTimeMs: resp.ProcessingTimeMs,\n        Query:            params.Query,\n        Source:           \"meilisearch\",\n    }, nil\n}\n```\n\n## MySQL fallback\n\nWhen Meilisearch is down (restart, OOM, maintenance), fall back to MySQL LIKE. It's slower and has no typo tolerance, but it keeps the API responding.\n\n```\n// search/fallback.go\npackage search\n\nimport (\n    \"database/sql\"\n    \"fmt\"\n    \"strings\"\n    \"time\"\n\n    _ \"github.com/go-sql-driver/mysql\"\n)\n\ntype MySQLFallback struct {\n    db *sql.DB\n}\n\nfunc NewMySQLFallback(dsn string) (*MySQLFallback, error) {\n    db, err := sql.Open(\"mysql\", dsn)\n    if err != nil {\n        return nil, err\n    }\n    db.SetMaxOpenConns(10)\n    db.SetConnMaxLifetime(5 * time.Minute)\n    return &MySQLFallback{db: db}, nil\n}\n\nfunc (m *MySQLFallback) Search(params SearchParams) (*SearchResult, error) {\n    query := \"%\" + strings.ReplaceAll(params.Query, \"%\", \"\\\\%\") + \"%\"\n\n    args := []interface{}{query, query}\n    where := \"WHERE (title LIKE ? OR content LIKE ?)\"\n\n    if params.Category != \"\" {\n        where += \" AND category = ?\"\n        args = append(args, params.Category)\n    }\n    if params.Difficulty != \"\" {\n        where += \" AND difficulty = ?\"\n        args = append(args, params.Difficulty)\n    }\n    if params.DocType != \"\" {\n        where += \" AND doc_type = ?\"\n        args = append(args, params.DocType)\n    }\n\n    // Count total\n    var total int64\n    countSQL := fmt.Sprintf(\"SELECT COUNT(*) FROM articles %s\", where)\n    _ = m.db.QueryRow(countSQL, args...).Scan(&total)\n\n    // Fetch page\n    args = append(args, params.Limit, params.Offset)\n    rows, err := m.db.Query(\n        fmt.Sprintf(`SELECT id, title, slug, category, difficulty, doc_type, published_at\n                     FROM articles %s ORDER BY published_at DESC LIMIT ? OFFSET ?`, where),\n        args...,\n    )\n    if err != nil {\n        return nil, err\n    }\n    defer rows.Close()\n\n    var hits []Article\n    for rows.Next() {\n        var a Article\n        if err := rows.Scan(&a.ID, &a.Title, &a.Slug,\n            &a.Category, &a.Difficulty, &a.DocType, &a.PublishedAt); err != nil {\n            continue\n        }\n        hits = append(hits, a)\n    }\n\n    return &SearchResult{\n        Hits:      hits,\n        TotalHits: total,\n        Query:     params.Query,\n        Source:    \"mysql_fallback\",\n    }, nil\n}\n```\n\n## HTTP handler\n\n```\n// handlers/search.go\npackage handlers\n\nimport (\n    \"encoding/json\"\n    \"log\"\n\n    \"github.com/gofiber/fiber/v2\"\n    \"search-api/search\"\n)\n\ntype SearchHandler struct {\n    meili    *search.MeiliSearcher\n    fallback *search.MySQLFallback\n}\n\nfunc NewSearchHandler(meili *search.MeiliSearcher, fallback *search.MySQLFallback) *SearchHandler {\n    return &SearchHandler{meili: meili, fallback: fallback}\n}\n\nfunc (h *SearchHandler) Handle(c *fiber.Ctx) error {\n    params := search.SearchParams{\n        Query:      c.Query(\"q\", \"\"),\n        Category:   c.Query(\"cat\", \"\"),\n        Difficulty: c.Query(\"diff\", \"\"),\n        DocType:    c.Query(\"type\", \"\"),\n        Limit:      int64(c.QueryInt(\"limit\", 10)),\n        Offset:     int64(c.QueryInt(\"offset\", 0)),\n    }\n\n    if len([]rune(params.Query)) > 200 {\n        return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{\n            \"error\": \"query too long\",\n        })\n    }\n\n    result, err := h.meili.Search(params)\n    if err != nil {\n        log.Printf(\"Meilisearch error: %v — falling back to MySQL\", err)\n        result, err = h.fallback.Search(params)\n        if err != nil {\n            return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{\n                \"error\": \"search unavailable\",\n            })\n        }\n    }\n\n    return c.JSON(result)\n}\n```\n\n## Wiring it up in main.go\n\n```\n// main.go\npackage main\n\nimport (\n    \"log\"\n    \"os\"\n\n    \"github.com/gofiber/fiber/v2\"\n    \"search-api/handlers\"\n    \"search-api/search\"\n)\n\nfunc main() {\n    meiliHost := os.Getenv(\"MEILI_HOST\")       // e.g. \"http://127.0.0.1:7700\"\n    meiliKey  := os.Getenv(\"MEILI_MASTER_KEY\")\n    mysqlDSN  := os.Getenv(\"MYSQL_DSN\")        // e.g. \"user:pass@tcp(127.0.0.1:3306)/dbname\"\n\n    meili, err := search.NewMeiliSearcher(meiliHost, meiliKey)\n    if err != nil {\n        log.Fatalf(\"Failed to connect to Meilisearch: %v\", err)\n    }\n\n    fallback, err := search.NewMySQLFallback(mysqlDSN)\n    if err != nil {\n        log.Fatalf(\"Failed to connect to MySQL: %v\", err)\n    }\n\n    app := fiber.New(fiber.Config{\n        ErrorHandler: func(c *fiber.Ctx, err error) error {\n            return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{\n                \"error\": err.Error(),\n            })\n        },\n    })\n\n    searchHandler := handlers.NewSearchHandler(meili, fallback)\n    app.Get(\"/api/search\", searchHandler.Handle)\n\n    // Reindex endpoint (protect with auth middleware in production)\n    app.Post(\"/admin/search/reindex\", func(c *fiber.Ctx) error {\n        var articles []search.Article\n        if err := c.BodyParser(&articles); err != nil {\n            return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{\"error\": err.Error()})\n        }\n        if err := meili.IndexDocuments(articles); err != nil {\n            return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{\"error\": err.Error()})\n        }\n        return c.JSON(fiber.Map{\"indexed\": len(articles)})\n    })\n\n    log.Fatal(app.Listen(\":4001\"))\n}\n```\n\n## Test it\n\n```\n# Basic search\ncurl \"http://localhost:4001/api/search?q=pentest\"\n\n# With filters\ncurl \"http://localhost:4001/api/search?q=NIS+2&cat=guide&diff=intermediate&limit=5\"\n\n# Typo tolerance in action\ncurl \"http://localhost:4001/api/search?q=penetartion+tesitng\"\n```\n\n## Key tuning decisions\n\n** SearchableAttributes order matters.** Meilisearch's\n\n`attribute`\n\nranking rule rewards matches in earlier attributes. Putting `title`\n\nfirst means a title match outranks a content match, which is almost always what you want.**Pagination cap.** The `MaxTotalHits: 10000`\n\nsetting prevents Meilisearch from doing expensive full-index scans for pagination deep into results. If users never go past page 20 at 10 results/page, set this to 200.\n\n**Fallback opacity.** The `source`\n\nfield in `SearchResult`\n\ntells the frontend (and your monitoring) when it's getting degraded results. Log every fallback occurrence — if Meilisearch is down and you're not paged, you won't know until users complain.\n\n**On-startup sync.** Pull all published articles from MySQL on startup and call `IndexDocuments`\n\n. For 10,000+ documents this takes 2–5 seconds; for 100,000+ batch in chunks of 5,000. This ensures your index is always consistent even after a Meilisearch restart.\n\nThis pattern — Meilisearch primary, MySQL LIKE fallback — gives you production-grade search with no single point of failure and a codebase any Go developer can reason about.", "url": "https://wpnews.pro/news/building-a-semantic-search-api-in-go-with-meilisearch", "canonical_source": "https://dev.to/ayinedjimi-consultants/building-a-semantic-search-api-in-go-with-meilisearch-17ck", "published_at": "2026-05-22 22:00:00+00:00", "updated_at": "2026-05-22 22:32:58.421237+00:00", "lang": "en", "topics": ["developer-tools", "cybersecurity", "open-source", "data"], "entities": ["Meilisearch", "Go", "Fiber", "MySQL", "AYI NEDJIMI Consultants"], "alternates": {"html": "https://wpnews.pro/news/building-a-semantic-search-api-in-go-with-meilisearch", "markdown": "https://wpnews.pro/news/building-a-semantic-search-api-in-go-with-meilisearch.md", "text": "https://wpnews.pro/news/building-a-semantic-search-api-in-go-with-meilisearch.txt", "jsonld": "https://wpnews.pro/news/building-a-semantic-search-api-in-go-with-meilisearch.jsonld"}}