{"id":1333,"date":"2024-08-31T12:25:04","date_gmt":"2024-08-31T04:25:04","guid":{"rendered":"https:\/\/www.fanyamin.com\/wordpress\/?p=1333"},"modified":"2024-08-31T16:18:50","modified_gmt":"2024-08-31T08:18:50","slug":"ask-llm-by-program","status":"publish","type":"post","link":"https:\/\/www.fanyamin.com\/wordpress\/?p=1333","title":{"rendered":"Ask LLM by program"},"content":{"rendered":"<h2>What's RAG<\/h2>\n<blockquote>\n<p>RAG is a technique for augmenting LLM knowledge with additional data.<br \/>\nRAG \u662f\u4e00\u79cd\u4ee5\u989d\u5916\u6570\u636e\u6765\u589e\u5f3a LLM \u77e5\u8bc6\u7684\u6280\u672f<\/p>\n<\/blockquote>\n<p>\u4ee5\u79c1\u6709\u6216\u8005\u8f83\u65b0\u7684\u77e5\u8bc6\u63d2\u5165\u5230\u7ed9\u6a21\u578b\u7684\u63d0\u793a\u7684\u8fc7\u7a0b\u5c31\u53eb\u68c0\u7d22\u589e\u5f3a\u751f\u6210<\/p>\n<h2>Concepts<\/h2>\n<h3>Indexing<\/h3>\n<ul>\n<li>\n<p>Load: First we need to load our data. This is done with Document Loaders.<br \/>\n\u8f7d\u5165\u6211\u4eec\u7684\u77e5\u8bc6<\/p>\n<\/li>\n<li>\n<p>Split: Text splitters break large Documents into smaller chunks. This is useful both for indexing data and for passing it in to a model, since large chunks are harder to search over and won't fit in a model's finite context window.<\/p>\n<\/li>\n<\/ul>\n<p>\u62c6\u5206\u5927\u6587\u6863\u5230\u5c0f\u5757\u6bb5\u843d<\/p>\n<ul>\n<li>Store: We need somewhere to store and index our splits, so that they can later be searched over. This is often done using a VectorStore and Embeddings model.<\/li>\n<\/ul>\n<p>\u5b58\u50a8\u548c\u7d22\u5f15\u4ee5\u4e0a\u62c6\u5206\u7684\u6bb5\u843d<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/www.fanyamin.com\/wordpress\/wp-content\/uploads\/2024\/08\/image-1725091916568.png\" alt=\"file\" \/><\/p>\n<h3>Retrieval and generation<\/h3>\n<ul>\n<li>Retrieve: Given a user input, relevant splits are retrieved from storage using a Retriever.<\/li>\n<\/ul>\n<p>\u7ed9\u51fa\u4e00\u4e2a\u7528\u6237\u8f93\u5165, \u901a\u8fc7 Retriver \u53d6\u51fa\u7684\u76f8\u5173\u7684\u62c6\u5206\u6bb5\u843d<\/p>\n<ul>\n<li>Generate: A ChatModel \/ LLM produces an answer using a prompt that includes the question and the retrieved data<\/li>\n<\/ul>\n<p>\u751f\u6210: \u4e00\u4e2a\u5bf9\u8bdd\u6a21\u578b\/\u5927\u8bed\u8a00\u6a21\u578b\u4f7f\u7528\u5305\u542b\u95ee\u9898\u548c\u6240\u68c0\u7d22\u51fa\u7684\u6570\u636e\u6240\u4ea7\u751f\u4e00\u4e2a\u56de\u7b54<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/www.fanyamin.com\/wordpress\/wp-content\/uploads\/2024\/08\/image-1725091978309.png\" alt=\"file\" \/><\/p>\n<h2>program to ask LLM<\/h2>\n<pre><code>\n#!\/usr\/bin\/env python3\nfrom openai import OpenAI\nfrom dotenv import load_dotenv\nimport os, sys\nimport argparse\nfrom loguru import logger\n\nload_dotenv()\nlogger.add(sys.stdout,\n           format=&quot;{time} {message}&quot;,\n           filter=&quot;client&quot;,\n           level=&quot;DEBUG&quot;)\n\nclass LlmAgent:\n\n    def __init__(self, api_key, base_url, model):\n        self._client = OpenAI(api_key=api_key,\n            base_url=base_url)\n        self._model = model\n\n    def ask_question(self, question: str):\n\n        response = self._client.chat.completions.create(\n            model=self._model,\n            messages=[\n                {&#039;role&#039;: &#039;user&#039;, &#039;content&#039;: question}\n            ],\n            stream=True\n        )\n\n        for chunk in response:\n            print(chunk.choices[0].delta.content, end=&#039;&#039;)\n\nif __name__ == &quot;__main__&quot;:\n\n    parser = argparse.ArgumentParser()\n    parser.add_argument(&#039;--action&#039;,&#039;-a&#039;, action=&#039;store&#039;, dest=&#039;action&#039;, help=&#039;specify action: ask|test&#039;)\n    parser.add_argument(&#039;--model&#039;, &#039;-m&#039;, dest=&#039;model&#039;, default=&quot;alibaba\/Qwen1.5-110B-Chat&quot;, help=&#039;LLM model&#039;)\n\n    args = parser.parse_args()\n    if (args.action == &quot;test&quot;):\n        agent = LlmAgent(os.getenv(&quot;LLM_API_KEY&quot;), os.getenv(&quot;LLM_BASE_URL&quot;), args.model)\n        agent.ask_question(&quot;how to parse PDF file by python?&quot;)\n    else:\n        print(&quot;example:&quot;)\n        print(&quot;\\t.\/ask_llm.py -a test&quot;)\n<\/code><\/pre>\n<h2>answer from LLM<\/h2>\n<p>NoneTo parse a PDF file in Python, you can use several libraries, one of the most popular being PyPDF2 and PDFMiner for text extraction. Keep in mind that parsing PDFs can be tricky due to differences in how PDFs are structured, and complex formatted PDFs might not retain perfect formatting when extracted.<\/p>\n<p><strong>1. Using PyPDF2 (for basic text extraction):<\/strong><\/p>\n<p>PyPDF2 is great for splitting, merging, and cropping PDFs, but it might not handle complex formatting very well.<\/p>\n<p>First, install PyPDF2:<\/p>\n<pre><code>pip install PyPDF2<\/code><\/pre>\n<p>Here's a basic code snippet to extract text from a PDF:<\/p>\n<pre><code class=\"language-python\">import PyPDF2\n\ndef extract_text_from_pdf(pdf_path):\n    text = &#039;&#039;\n    with open(pdf_path, &#039;rb&#039;) as file:\n        pdf_reader = PyPDF2.PdfFileReader(file)\n        for page_num in range(pdf_reader.getNumPages()):\n            text += pdf_reader.getPage(page_num).extractText()\n    return text\n\npdf_path = &#039;your_pdf_file.pdf&#039;\ntext = extract_text_from_pdf(pdf_path)\nprint(text)<\/code><\/pre>\n<p><strong>2. Using PDFMiner (recommended for accurate text extraction):<\/strong><\/p>\n<p>PDFMiner is better suited for extracting text accurately, including layout information.<\/p>\n<p>First, install PDFMiner using pip:<\/p>\n<pre><code class=\"language-bash\">pip install pdfminer.six  # Use this for Python 3<\/code><\/pre>\n<p>Here's a simple example of how to use PDFMiner to extract text:<\/p>\n<pre><code class=\"language-python\">from pdfminer.high_level import extract_text\n\ndef extract_text_pdfminer(pdf_path):\n    text = extract_text(pdf_path)\n    return text\n\npdf_path = &#039;your_pdf_file.pdf&#039;\ntext = extract_text_pdfminer(pdf_path)\nprint(text)<\/code><\/pre>\n<p>For more advanced usage, such as extracting specific elements, positions, or dealing with encrypted PDFs, consult the respective libraries\u2019 documentation. PDFMiner, in particular, allows for a lot of customization through lower-level APIs, which can be helpful in handling complex PDF structures.None%<\/p>\n","protected":false},"excerpt":{"rendered":"<p>What&#8217;s RAG RAG is a technique for augmenting LLM knowledge with additional data. RAG \u662f\u4e00\u79cd\u4ee5\u989d\u5916\u6570\u636e\u6765\u589e\u5f3a LLM \u77e5\u8bc6\u7684\u6280\u672f \u4ee5\u79c1\u6709\u6216\u8005\u8f83\u65b0\u7684\u77e5\u8bc6\u63d2\u5165\u5230\u7ed9\u6a21\u578b\u7684\u63d0\u793a\u7684\u8fc7\u7a0b\u5c31\u53eb\u68c0\u7d22\u589e\u5f3a\u751f\u6210 Concepts Indexing Load: First we need to load our data. This is done with Document Loaders. \u8f7d\u5165\u6211\u4eec\u7684\u77e5\u8bc6 Split: Text splitters break large Documents into smaller chunks. This is useful both for indexing data and for passing it [&hellip;] <a class=\"read-more\" href=\"https:\/\/www.fanyamin.com\/wordpress\/?p=1333\" title=\"Permanent Link to: Ask LLM by program\">&rarr;Read&nbsp;more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"class_list":["post-1333","post","type-post","status-publish","format-standard","hentry","category-5"],"_links":{"self":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/1333"}],"collection":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1333"}],"version-history":[{"count":5,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/1333\/revisions"}],"predecessor-version":[{"id":1340,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/1333\/revisions\/1340"}],"wp:attachment":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1333"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1333"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1333"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}