{"id":1303,"date":"2024-08-26T23:16:34","date_gmt":"2024-08-26T15:16:34","guid":{"rendered":"https:\/\/www.fanyamin.com\/wordpress\/?p=1303"},"modified":"2024-08-27T11:14:56","modified_gmt":"2024-08-27T03:14:56","slug":"fastapi-with-celery","status":"publish","type":"post","link":"https:\/\/www.fanyamin.com\/wordpress\/?p=1303","title":{"rendered":"fastapi with celery"},"content":{"rendered":"<h2>Overview<\/h2>\n<p>\u8fd9\u5176\u5b9e\u662f\u5178\u578b\u7684\u53d1\u5e03\u8ba2\u9605\u6a21\u5f0f, \u751f\u4ea7\u8005\u53d1\u5e03\u4efb\u52a1\u5230\u4efb\u52a1\u961f\u5217\u4e2d, \u6d88\u8d39\u8005\u4ece\u4efb\u52a1\u961f\u5217\u4e2d\u6d88\u8d39\u4efb\u52a1.<br \/>\n\u8fd9\u91cc\u7684\u4efb\u52a1\u961f\u5217\u662f Redis, \u4e5f\u53ef\u4ee5\u4f7f\u7528 RabbitMQ<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/www.fanyamin.com\/wordpress\/wp-content\/uploads\/2024\/08\/image-1724725602362.png\" alt=\"file\" \/><\/p>\n<h2>Quick start<\/h2>\n<pre><code>from fastapi import FastAPI\nfrom pydantic import BaseModel\n\napp = FastAPI()\n\nclass Item(BaseModel):\n    item_id: int\n    name: str\n    description: str = None\n\n@app.get(&quot;\/items\/{item_id}&quot;, response_model=Item)\nasync def get_item(item_id: int):\n    return {&quot;item_id&quot;: item_id, &quot;name&quot;: &quot;Some item&quot;, &quot;description&quot;: &quot;Description of the item&quot;}<\/code><\/pre>\n<h2>start cerely by docker compose<\/h2>\n<pre><code>version: &#039;3.8&#039;\n\nservices:\n  web:\n    build:\n      context: .\n      dockerfile: .\/compose\/local\/fastapi\/Dockerfile\n    image: fastapi_celery_example_web\n    # &#039;\/start&#039; is the shell script used to run the service\n    command: \/start\n    # this volume is used to map the files and folders on the host to the container\n    # so if we change code on the host, code in the docker container will also be changed\n    volumes:\n      - .:\/app\n    ports:\n      - 8010:8000\n    env_file:\n      - .env\/.dev-sample\n    depends_on:\n      - redis\n      - db\n\n  db:\n    image: postgres:16-alpine\n    volumes:\n      - postgres_data:\/var\/lib\/postgresql\/data\/\n    environment:\n      - POSTGRES_DB=fastapi_celery\n      - POSTGRES_USER=fastapi_celery\n      - POSTGRES_PASSWORD=fastapi_celery\n\n  redis:\n    image: redis:7-alpine\n\n  celery_worker:\n    build:\n      context: .\n      dockerfile: .\/compose\/local\/fastapi\/Dockerfile\n    image: fastapi_celery_example_celery_worker\n    command: \/start-celeryworker\n    volumes:\n      - .:\/app\n    env_file:\n      - .env\/.dev-sample\n    depends_on:\n      - redis\n      - db\n\n  celery_beat:\n    build:\n      context: .\n      dockerfile: .\/compose\/local\/fastapi\/Dockerfile\n    image: fastapi_celery_example_celery_beat\n    command: \/start-celerybeat\n    volumes:\n      - .:\/app\n    env_file:\n      - .env\/.dev-sample\n    depends_on:\n      - redis\n      - db\n\n  flower:\n    build:\n      context: .\n      dockerfile: .\/compose\/local\/fastapi\/Dockerfile\n    image: fastapi_celery_example_celery_flower\n    command: \/start-flower\n    volumes:\n      - .:\/app\n    env_file:\n      - .env\/.dev-sample\n    ports:\n      - 5557:5555\n    depends_on:\n      - redis\n      - db\nvolumes:\n  postgres_data:<\/code><\/pre>\n<h2>folder structure example<\/h2>\n<pre><code>\u251c\u2500\ud83d\udcc1 backend--------------- # Backend\n\u2502\u2003\u251c\u2500\ud83d\udcc1 alembic------------- # DB migration\n\u2502\u2003\u251c\u2500\ud83d\udcc1 app----------------- # Application\n\u2502\u2003\u2502\u2003\u251c\u2500\ud83d\udcc1 admin------------- # System admin\n\u2502\u2003\u2502\u2003\u2502\u2003\u251c\u2500\ud83d\udcc1 api------------- # Interface\n\u2502\u2003\u2502\u2003\u2502\u2003\u251c\u2500\ud83d\udcc1 crud------------ # CRUD\n\u2502\u2003\u2502\u2003\u2502\u2003\u251c\u2500\ud83d\udcc1 model----------- # SQLA model\n\u2502\u2003\u2502\u2003\u2502\u2003\u251c\u2500\ud83d\udcc1 schema---------- # Data transmit\n\u2502\u2003\u2502\u2003\u2502\u2003\u251c\u2500\ud83d\udcc1 service--------- # Service\n\u2502\u2003\u2502\u2003\u2502\u2003\u2514\u2500\ud83d\udcc1 tests----------- # Pytest\n\u2502\u2003\u2502\u2003\u251c\u2500\ud83d\udcc1 generator--------- # Code generate\n\u2502\u2003\u2502\u2003\u2514\u2500\ud83d\udcc1 task-------------- # Celery task\n\u2502\u2003\u251c\u2500\ud83d\udcc1 common-------------- # public resources\n\u2502\u2003\u251c\u2500\ud83d\udcc1 core---------------- # Core configuration\n\u2502\u2003\u251c\u2500\ud83d\udcc1 database------------ # Database connection\n\u2502\u2003\u251c\u2500\ud83d\udcc1 log----------------- # Log\n\u2502\u2003\u251c\u2500\ud83d\udcc1 middleware---------- # Middlewares\n\u2502\u2003\u251c\u2500\ud83d\udcc1 scripts------------- # Scripts\n\u2502\u2003\u251c\u2500\ud83d\udcc1 sql----------------- # SQL files\n\u2502\u2003\u251c\u2500\ud83d\udcc1 static-------------- # Static files\n\u2502\u2003\u251c\u2500\ud83d\udcc1 templates----------- # Template files\n\u2502\u2003\u2514\u2500\ud83d\udcc1 utils--------------- # Toolkit\n\u2514\u2500\ud83d\udcc1 deploy---------------- # Server deployment\n<\/code><\/pre>\n<h2>Dockerfile<\/h2>\n<pre><code>FROM python3.10-slim\n\nWORKDIR \/app\n\nCOPY requirements.txt .\n\nRUN pip install -r requirements.txt\n\nCOPY . .\n\nEXPOSE 8000\n\nCMD [&quot;uvicorn&quot;, &quot;main:app&quot;]<\/code><\/pre>\n<h2>celery<\/h2>\n<pre><code>celery -A app.task.celery worker -l info\n\n# Scheduled tasks (optional)\ncelery -A app.task.celery beat -l info\n\n# Web monitor (optional)\ncelery -A app.task.celery flower --port=8555 --basic-auth=admin:123456<\/code><\/pre>\n<h2>snippets<\/h2>\n<ul>\n<li>config.py<\/li>\n<\/ul>\n<pre><code>import os\nimport pathlib\nfrom functools import lru_cache\n\nclass BaseConfig:\n    BASE_DIR: pathlib.Path = pathlib.Path(__file__).parent.parent\n\n    DATABASE_URL: str = os.environ.get(&quot;DATABASE_URL&quot;, f&quot;sqlite:\/\/\/{BASE_DIR}\/db.sqlite3&quot;)\n    DATABASE_CONNECT_DICT: dict = {}\n\nclass DevelopmentConfig(BaseConfig):\n    pass\n\nclass ProductionConfig(BaseConfig):\n    pass\n\nclass TestingConfig(BaseConfig):\n    pass\n\n@lru_cache()\ndef get_settings():\n    config_cls_dict = {\n        &quot;development&quot;: DevelopmentConfig,\n        &quot;production&quot;: ProductionConfig,\n        &quot;testing&quot;: TestingConfig\n    }\n\n    config_name = os.environ.get(&quot;FASTAPI_CONFIG&quot;, &quot;development&quot;)\n    config_cls = config_cls_dict[config_name]\n    return config_cls()\n\nsettings = get_settings()<\/code><\/pre>\n<ul>\n<li>database.py<\/li>\n<\/ul>\n<pre><code>from sqlalchemy import create_engine\nfrom sqlalchemy.orm import sessionmaker, declarative_base\n\nfrom project.config import settings\n\n# https:\/\/fastapi.tiangolo.com\/tutorial\/sql-databases\/#create-the-sqlalchemy-engine\nengine = create_engine(\n    settings.DATABASE_URL, connect_args=settings.DATABASE_CONNECT_DICT\n)\nSessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)\n\nBase = declarative_base()<\/code><\/pre>\n<h2>reference<\/h2>\n<ul>\n<li><a href=\"https:\/\/testdriven.io\/blog\/moving-from-flask-to-fastapi\/\">https:\/\/testdriven.io\/blog\/moving-from-flask-to-fastapi\/<\/a><\/li>\n<li><a href=\"https:\/\/testdriven.io\/courses\/fastapi-celery\/app-factory\/\">https:\/\/testdriven.io\/courses\/fastapi-celery\/app-factory\/<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Overview \u8fd9\u5176\u5b9e\u662f\u5178\u578b\u7684\u53d1\u5e03\u8ba2\u9605\u6a21\u5f0f, \u751f\u4ea7\u8005\u53d1\u5e03\u4efb\u52a1\u5230\u4efb\u52a1\u961f\u5217\u4e2d, \u6d88\u8d39\u8005\u4ece\u4efb\u52a1\u961f\u5217\u4e2d\u6d88\u8d39\u4efb\u52a1. \u8fd9\u91cc\u7684\u4efb\u52a1\u961f\u5217\u662f Redis, \u4e5f\u53ef\u4ee5\u4f7f\u7528 RabbitMQ Quick start from fastapi import FastAPI from pydantic import BaseModel app = FastAPI() class Item(BaseModel): item_id: int name: str description: str = None @app.get(&quot;\/items\/{item_id}&quot;, response_model=Item) async def get_item(item_id: int): return {&quot;item_id&quot;: item_id, &quot;name&quot;: &quot;Some item&quot;, &quot;description&quot;: &quot;Description of the item&quot;} start cerely by docker compose version: &#039;3.8&#039; [&hellip;] <a class=\"read-more\" href=\"https:\/\/www.fanyamin.com\/wordpress\/?p=1303\" title=\"Permanent Link to: fastapi with celery\">&rarr;Read&nbsp;more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"class_list":["post-1303","post","type-post","status-publish","format-standard","hentry","category-5"],"_links":{"self":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/1303"}],"collection":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1303"}],"version-history":[{"count":10,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/1303\/revisions"}],"predecessor-version":[{"id":1319,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/1303\/revisions\/1319"}],"wp:attachment":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1303"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1303"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1303"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}