{"id":2069,"date":"2025-05-27T18:04:19","date_gmt":"2025-05-27T10:04:19","guid":{"rendered":"https:\/\/www.fanyamin.com\/wordpress\/?p=2069"},"modified":"2025-05-27T18:04:39","modified_gmt":"2025-05-27T10:04:39","slug":"how-to-build-local-knowledge-base-by-llm","status":"publish","type":"post","link":"https:\/\/www.fanyamin.com\/wordpress\/?p=2069","title":{"rendered":"how to build local knowledge base by LLM 1?"},"content":{"rendered":"<p>we can build a <strong>local knowledge base powered by RAG (Retrieval-Augmented Generation)<\/strong> using:<\/p>\n<ul>\n<li>\ud83d\udd0d <strong>ChromaDB<\/strong> (embedded vector DB)<\/li>\n<li>\ud83e\udde0 <strong>Ollama<\/strong> (local LLM: Mistral, LLaMA, etc.)<\/li>\n<li>\u270d\ufe0f <strong>Go code<\/strong> to ingest documents, embed chunks, and query with context<\/li>\n<\/ul>\n<hr \/>\n<h2>\u2705 Prerequisites<\/h2>\n<p>Install the following tools:<\/p>\n<pre><code class=\"language-bash\"># Chroma as a Python backend server\npip install chromadb\n\n# Ollama for running local LLMs (like mistral)\nbrew install ollama      # or follow: https:\/\/ollama.com\/download\nollama run mistral       # or other models\n\n# Go packages\ngo get github.com\/go-resty\/resty\/v2\ngo get github.com\/google\/uuid<\/code><\/pre>\n<hr \/>\n<h2>\ud83d\uddc2\ufe0f Project Structure<\/h2>\n<pre><code>go-local-rag\/\n\u251c\u2500\u2500 main.go\n\u251c\u2500\u2500 embedder.go       # Calls embedding model (local or OpenAI)\n\u251c\u2500\u2500 chroma_client.go  # Handles Chroma DB operations\n\u251c\u2500\u2500 prompt.go         # Constructs LLM prompts\n\u251c\u2500\u2500 docs\/\n\u2502   \u2514\u2500\u2500 sample.txt<\/code><\/pre>\n<hr \/>\n<h2>\ud83e\udde0 Step-by-Step Sample: <code>main.go<\/code><\/h2>\n<pre><code class=\"language-go\">package main\n\nimport (\n    &quot;fmt&quot;\n    &quot;log&quot;\n    &quot;strings&quot;\n\n    &quot;your_project\/chroma&quot;\n    &quot;your_project\/embedder&quot;\n    &quot;your_project\/prompt&quot;\n)\n\nfunc main() {\n    docs := []string{\n        &quot;Go is a statically typed, compiled programming language designed at Google.&quot;,\n        &quot;It is syntactically similar to C, but with memory safety, garbage collection, and CSP-style concurrency.&quot;,\n    }\n\n    \/\/ Step 1: Embed and store in ChromaDB\n    for _, doc := range docs {\n        chunks := strings.Split(doc, &quot;. &quot;) \/\/ naive splitter\n        for _, chunk := range chunks {\n            vec := embedder.EmbedText(chunk)\n            chroma.Upsert(&quot;golang-knowledge&quot;, chunk, vec)\n        }\n    }\n\n    \/\/ Step 2: User query\n    query := &quot;What is Go and who designed it?&quot;\n    qvec := embedder.EmbedText(query)\n\n    \/\/ Step 3: Retrieve relevant docs\n    topChunks := chroma.Query(&quot;golang-knowledge&quot;, qvec)\n\n    \/\/ Step 4: Compose prompt and send to LLM (Ollama)\n    finalPrompt := prompt.Build(query, topChunks)\n    resp := prompt.CallOllama(&quot;mistral&quot;, finalPrompt)\n\n    fmt.Println(&quot;\ud83d\udd0e Answer:\\n&quot;, resp)\n}<\/code><\/pre>\n<hr \/>\n<h2>\ud83d\udd0c Sample: <code>embedder.go<\/code> (Ollama embedding or OpenAI)<\/h2>\n<pre><code class=\"language-go\">package embedder\n\nimport (\n    &quot;bytes&quot;\n    &quot;encoding\/json&quot;\n    &quot;log&quot;\n    &quot;net\/http&quot;\n)\n\nfunc EmbedText(text string) []float64 {\n    payload := map[string]string{&quot;model&quot;: &quot;nomic-embed-text&quot;, &quot;prompt&quot;: text}\n    body, _ := json.Marshal(payload)\n\n    resp, err := http.Post(&quot;http:\/\/localhost:11434\/api\/embeddings&quot;, &quot;application\/json&quot;, bytes.NewBuffer(body))\n    if err != nil {\n        log.Fatal(err)\n    }\n    defer resp.Body.Close()\n\n    var result struct {\n        Embedding []float64 `json:&quot;embedding&quot;`\n    }\n    json.NewDecoder(resp.Body).Decode(&amp;result)\n    return result.Embedding\n}<\/code><\/pre>\n<hr \/>\n<h2>\ud83e\uddf1 Sample: <code>chroma_client.go<\/code><\/h2>\n<pre><code class=\"language-go\">package chroma\n\nimport (\n    &quot;bytes&quot;\n    &quot;encoding\/json&quot;\n    &quot;fmt&quot;\n    &quot;log&quot;\n    &quot;net\/http&quot;\n\n    &quot;github.com\/google\/uuid&quot;\n)\n\ntype ChromaDoc struct {\n    ID      string\n    Content string\n    Vector  []float64\n}\n\nvar chromaURL = &quot;http:\/\/localhost:8000&quot;\n\nfunc Upsert(collection string, content string, vec []float64) {\n    payload := map[string]interface{}{\n        &quot;ids&quot;:     []string{uuid.New().String()},\n        &quot;documents&quot;: []string{content},\n        &quot;embeddings&quot;: [][]float64{vec},\n    }\n    data, _ := json.Marshal(payload)\n\n    url := fmt.Sprintf(&quot;%s\/api\/v1\/collections\/%s\/upsert&quot;, chromaURL, collection)\n    _, err := http.Post(url, &quot;application\/json&quot;, bytes.NewReader(data))\n    if err != nil {\n        log.Fatal(err)\n    }\n}\n\nfunc Query(collection string, vec []float64) []string {\n    payload := map[string]interface{}{\n        &quot;embedding&quot;: vec,\n        &quot;n_results&quot;: 3,\n    }\n    data, _ := json.Marshal(payload)\n\n    url := fmt.Sprintf(&quot;%s\/api\/v1\/collections\/%s\/query&quot;, chromaURL, collection)\n    resp, err := http.Post(url, &quot;application\/json&quot;, bytes.NewReader(data))\n    if err != nil {\n        log.Fatal(err)\n    }\n    defer resp.Body.Close()\n\n    var result struct {\n        Documents [][]string `json:&quot;documents&quot;`\n    }\n    json.NewDecoder(resp.Body).Decode(&amp;result)\n    if len(result.Documents) &gt; 0 {\n        return result.Documents[0]\n    }\n    return []string{}\n}<\/code><\/pre>\n<hr \/>\n<h2>\ud83e\udde0 Sample: <code>prompt.go<\/code><\/h2>\n<pre><code class=\"language-go\">package prompt\n\nimport (\n    &quot;bytes&quot;\n    &quot;encoding\/json&quot;\n    &quot;fmt&quot;\n    &quot;net\/http&quot;\n    &quot;strings&quot;\n)\n\nfunc Build(query string, contexts []string) string {\n    contextStr := strings.Join(contexts, &quot;\\n&quot;)\n    return fmt.Sprintf(&quot;Answer the question based on the context:\\n\\n%s\\n\\nQuestion: %s&quot;, contextStr, query)\n}\n\nfunc CallOllama(model, prompt string) string {\n    payload := map[string]string{\n        &quot;model&quot;:  model,\n        &quot;prompt&quot;: prompt,\n    }\n    body, _ := json.Marshal(payload)\n\n    resp, err := http.Post(&quot;http:\/\/localhost:11434\/api\/generate&quot;, &quot;application\/json&quot;, bytes.NewBuffer(body))\n    if err != nil {\n        panic(err)\n    }\n    defer resp.Body.Close()\n\n    var out struct {\n        Response string `json:&quot;response&quot;`\n    }\n    json.NewDecoder(resp.Body).Decode(&amp;out)\n    return out.Response\n}<\/code><\/pre>\n<hr \/>\n<h2>\ud83e\uddea Running It<\/h2>\n<ol>\n<li>\n<p>Start ChromaDB:<\/p>\n<pre><code class=\"language-bash\">chroma run --path .\/chroma-data<\/code><\/pre>\n<\/li>\n<li>\n<p>Start Ollama:<\/p>\n<pre><code class=\"language-bash\">ollama run mistral<\/code><\/pre>\n<\/li>\n<li>\n<p>Run your Go program:<\/p>\n<pre><code class=\"language-bash\">go run main.go<\/code><\/pre>\n<\/li>\n<\/ol>\n<hr \/>\n<h2>\ud83d\udccc Next Steps<\/h2>\n<ul>\n<li>Add PDF\/Markdown ingestion using <code>github.com\/ledongthuc\/pdf<\/code> or <code>blackfriday<\/code><\/li>\n<li>Switch to LlamaIndex or LangChain via REST API for rapid prototyping<\/li>\n<li>Store user feedback to improve answers<\/li>\n<li>Build a simple web UI with Go + Svelte\/React<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>we can build a local knowledge base powered by RAG (Retrieval-Augmented Generation) using: \ud83d\udd0d ChromaDB (embedded vector DB) \ud83e\udde0 Ollama (local LLM: Mistral, LLaMA, etc.) \u270d\ufe0f Go code to ingest documents, embed chunks, and query with context \u2705 Prerequisites Install the following tools: # Chroma as a Python backend server pip install chromadb # Ollama [&hellip;] <a class=\"read-more\" href=\"https:\/\/www.fanyamin.com\/wordpress\/?p=2069\" title=\"Permanent Link to: how to build local knowledge base by LLM 1?\">&rarr;Read&nbsp;more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"class_list":["post-2069","post","type-post","status-publish","format-standard","hentry","category-5"],"_links":{"self":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/2069"}],"collection":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2069"}],"version-history":[{"count":2,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/2069\/revisions"}],"predecessor-version":[{"id":2071,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/2069\/revisions\/2071"}],"wp:attachment":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2069"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2069"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2069"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}