{"id":2072,"date":"2025-05-27T18:05:08","date_gmt":"2025-05-27T10:05:08","guid":{"rendered":"https:\/\/www.fanyamin.com\/wordpress\/?p=2072"},"modified":"2025-05-27T18:05:08","modified_gmt":"2025-05-27T10:05:08","slug":"build-local-knowledge-base-2","status":"publish","type":"post","link":"https:\/\/www.fanyamin.com\/wordpress\/?p=2072","title":{"rendered":"build local knowledge base 2"},"content":{"rendered":"

Building a Local Knowledge Base and Agent with LLM<\/h1>\n

Here's a detailed guide on how to create a local knowledge base from your files and build an agent that can interact with this knowledge.<\/p>\n

Part 1: Building a Local Knowledge Base<\/h2>\n

Step 1: Set Up Your Environment<\/h3>\n
# Create a virtual environment\npython -m venv llm_kb_env\nsource llm_kb_env\/bin\/activate  # On Windows: llm_kb_env\\Scripts\\activate\n\n# Install necessary packages\npip install langchain langchain_community pypdf python-docx sentence-transformers chromadb\npip install llama-index openai faiss-cpu tiktoken<\/code><\/pre>\n
Step 2: Organize Your Documents<\/h3>\n
    \n
  1. Create a directory for your document collection\n
    mkdir -p data\/documents<\/code><\/pre>\n<\/li>\n
  2. Place your files (code, txt, pdf, etc.) in this directory<\/li>\n<\/ol>\n
    Step 3: Document Loading and Processing<\/h3>\n
    from langchain.document_loaders import (\n    TextLoader,\n    PyPDFLoader,\n    DirectoryLoader,\n    UnstructuredMarkdownLoader,\n    CSVLoader\n)\nfrom langchain.text_splitter import RecursiveCharacterTextSplitter\n\n# Configure loaders for different file types\nloaders = {\n    "txt": DirectoryLoader("data\/documents\/", glob="**\/*.txt", loader_cls=TextLoader),\n    "pdf": DirectoryLoader("data\/documents\/", glob="**\/*.pdf", loader_cls=PyPDFLoader),\n    "md": DirectoryLoader("data\/documents\/", glob="**\/*.md", loader_cls=UnstructuredMarkdownLoader),\n    "csv": DirectoryLoader("data\/documents\/", glob="**\/*.csv", loader_cls=CSVLoader),\n    # Add more file types as needed\n}\n\n# Load documents\ndocuments = []\nfor loader_type, loader in loaders.items():\n    try:\n        documents.extend(loader.load())\n        print(f"Loaded {loader_type} documents")\n    except Exception as e:\n        print(f"Error loading {loader_type} documents: {e}")\n\n# Split documents into chunks\ntext_splitter = RecursiveCharacterTextSplitter(\n    chunk_size=1000,\n    chunk_overlap=200,\n    length_function=len\n)\nchunks = text_splitter.split_documents(documents)\nprint(f"Split into {len(chunks)} chunks")<\/code><\/pre>\n
    Step 4: Create Vector Embeddings<\/h3>\n
    from langchain.embeddings import HuggingFaceEmbeddings\nfrom langchain.vectorstores import Chroma, FAISS\n\n# Initialize embeddings\nembedding_model = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")\n\n# Option 1: Create a ChromaDB vector store\nvectorstore = Chroma.from_documents(\n    documents=chunks,\n    embedding=embedding_model,\n    persist_directory="data\/chroma_db"\n)\nvectorstore.persist()  # Save to disk\n\n# Option 2: Create a FAISS vector store (in-memory, can be saved)\nfaiss_index = FAISS.from_documents(chunks, embedding_model)\nfaiss_index.save_local("data\/faiss_index")<\/code><\/pre>\n
    Part 2: Building an Agent Based on the Knowledge Base<\/h2>\n
    Step 5: Set Up Retrieval System<\/h3>\n
    from langchain.retrievers import ContextualCompressionRetriever\nfrom langchain.retrievers.document_compressors import LLMChainExtractor\nfrom langchain.chat_models import ChatOpenAI\nfrom langchain.chains import RetrievalQA\n\n# Load your vectorstore\nvectorstore = Chroma(persist_directory="data\/chroma_db", embedding_function=embedding_model)\n\n# Create a retriever\nretriever = vectorstore.as_retriever(search_type="similarity", search_kwargs={"k": 5})\n\n# Optional: Add contextual compression for better results\nllm = ChatOpenAI(temperature=0)  # You can use local models too\ncompressor = LLMChainExtractor.from_llm(llm)\ncompression_retriever = ContextualCompressionRetriever(\n    base_compressor=compressor,\n    base_retriever=retriever\n)\n\n# Create a question-answering chain\nqa_chain = RetrievalQA.from_chain_type(\n    llm=llm,\n    chain_type="stuff",  # Other options: "map_reduce", "refine"\n    retriever=retriever,\n    return_source_documents=True,\n    verbose=True\n)<\/code><\/pre>\n
    Step 6: Create an Agent with Tools<\/h3>\n
    from langchain.agents import Tool, initialize_agent, AgentType\nfrom langchain.memory import ConversationBufferMemory\n\n# Define a search tool using our QA chain\nsearch_tool = Tool(\n    name="Document Search",\n    func=qa_chain.run,\n    description="Useful for searching information in your documents. Input should be a question."\n)\n\n# Optional: Add a calculation tool\nfrom langchain.tools import Tool\nfrom langchain.chains import LLMMathChain\n\nllm_math_chain = LLMMathChain.from_llm(llm=llm, verbose=True)\nmath_tool = Tool(\n    name="Calculator",\n    func=llm_math_chain.run,\n    description="Useful for performing calculations. Input should be a mathematical expression."\n)\n\n# Create a memory system\nmemory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)\n\n# Initialize the agent\nagent = initialize_agent(\n    tools=[search_tool, math_tool],\n    llm=llm,\n    agent=AgentType.CHAT_CONVERSATIONAL_REACT_DESCRIPTION,\n    verbose=True,\n    memory=memory\n)<\/code><\/pre>\n
    Step 7: Create a Simple Interface<\/h3>\n
    def query_agent(query):\n    try:\n        response = agent.run(input=query)\n        return response\n    except Exception as e:\n        return f"Error: {str(e)}"\n\n# Interactive loop\nif __name__ == "__main__":\n    print("Welcome to your document assistant! Type 'exit' to quit.")\n    while True:\n        query = input("Question: ")\n        if query.lower() == 'exit':\n            break\n        response = query_agent(query)\n        print(f"Answer: {response}")<\/code><\/pre>\n
    Part 3: Using Local LLMs (Optional)<\/h2>\n
    If you want to keep everything local, replace the OpenAI model with a local model:<\/p>\n
    from langchain.llms import HuggingFacePipeline\nimport torch\nfrom transformers import AutoTokenizer, AutoModelForCausalLM, pipeline\n\n# Load local model\nmodel_id = "TheBloke\/Llama-2-7B-Chat-GGUF"  # Choose an appropriate model\ntokenizer = AutoTokenizer.from_pretrained(model_id)\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_id, \n    device_map="auto",\n    torch_dtype=torch.float16,\n    load_in_8bit=True  # For memory efficiency\n)\n\n# Create a pipeline\npipe = pipeline(\n    "text-generation",\n    model=model,\n    tokenizer=tokenizer,\n    max_length=2048,\n    temperature=0.3,\n    top_p=0.95,\n    repetition_penalty=1.15\n)\n\n# Create LLM\nlocal_llm = HuggingFacePipeline(pipeline=pipe)\n\n# Replace the OpenAI model with your local model in the QA chain and agent<\/code><\/pre>\n
    Part 4: Advanced Features<\/h2>\n
    Adding a Web Search Tool<\/h3>\n
    from langchain.utilities import GoogleSearchAPIWrapper\n\nsearch = GoogleSearchAPIWrapper()\nweb_search_tool = Tool(\n    name="Web Search",\n    description="Search Google for recent or factual information.",\n    func=search.run\n)\n\n# Add to your agent's tools list<\/code><\/pre>\n
    Metadata Filtering<\/h3>\n
    # When querying the vectorstore\nmetadata_filter = {"source": "important_document.pdf"}\nfiltered_docs = vectorstore.similarity_search(\n    "your query", \n    k=5, \n    filter=metadata_filter\n)<\/code><\/pre>\n
    Evaluation<\/h3>\n
    from langchain.evaluation.qa import QAEvalChain\n\n# Create some examples\nexamples = [\n    {"query": "What is X?", "answer": "X is Y according to the documents."}\n    # Add more examples\n]\n\n# Create an evaluation chain\neval_chain = QAEvalChain.from_llm(llm)\ngraded_outputs = eval_chain.evaluate(examples, [qa_chain.run(eg["query"]) for eg in examples])<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"
    Building a Local Knowledge Base and Agent with LLM Here’s a detailed guide on how to create a local knowledge base from your files and build an agent that can interact with this knowledge. Part 1: Building a Local Knowledge Base Step 1: Set Up Your Environment # Create a virtual environment python -m venv […] →Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"class_list":["post-2072","post","type-post","status-publish","format-standard","hentry","category-5"],"_links":{"self":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/2072"}],"collection":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2072"}],"version-history":[{"count":1,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/2072\/revisions"}],"predecessor-version":[{"id":2073,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/2072\/revisions\/2073"}],"wp:attachment":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2072"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2072"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2072"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}