# Sync — ML service (FastAPI)

Standalone **FastAPI** service that performs **zero-shot** predictions for:

1. **Influencer profiles** — niche probability distribution + popularity-style tier distribution from bio, captions, hashtags, and metrics.  
2. **Business briefs** — niche + tier-style probabilities from free-text description and budget range.

Inference uses **Groq’s hosted LLaMA** models (OpenAI-compatible chat completions). No local GPU or PyTorch runtime is required for normal operation.

The **Django backend** calls this service over HTTP (`ML_SERVICE_URL`); see `backend/core/services/ml_client.py`.

---

## Features

- **JSON-first prompts** — responses parsed and normalized to probability maps.
- **Health endpoint** — cheap check for operations / load balancers.
- **CORS** — configurable allowlist for browser tools (primary consumer is the backend, server-to-server).
- **Separation from Django** — scale or deploy the ML tier independently.

---

## Repository layout

```
ml_service/
├── run.py                 # uvicorn entry (uses app.config settings)
├── requirements.txt
├── .env                   # Create locally (see below)
├── app/
│   ├── main.py            # FastAPI app factory & router include
│   ├── config.py          # pydantic-settings: GROQ_*, PORT, CORS, etc.
│   ├── constants.py       # Niche labels, tier labels, prompt constraints
│   ├── dependencies.py  # FastAPI deps (if any)
│   ├── models/
│   │   ├── base_model.py  # Groq client wrapper (LLaMAModel)
│   │   └── predictor.py   # Orchestrates prompts + parsing
│   ├── routers/
│   │   ├── health.py
│   │   └── predict.py     # /api/v1/predict/*
│   ├── schemas/           # Pydantic request/response models
│   └── utils/
│       ├── prompts.py     # Prompt templates
│       ├── parser.py      # JSON extraction / normalization
│       └── pricing.py     # Helpers used in evaluation / parity tooling
└── evaluation/            # Offline evaluation scripts & datasets (optional)
```

> Legacy comments in `requirements.txt` refer to optional local `torch` / `transformers`; **production path is Groq-only**.

---

## Configuration (`.env`)

Create **`ml_service/.env`** (loaded by `pydantic-settings` in `app/config.py`):

| Variable | Description |
|----------|-------------|
| **`GROQ_API_KEY`** | **Required** for real inference. |
| `GROQ_MODEL` | Groq model id (default in code may be e.g. `llama-3.2-3b-preview` — check `app/config.py`). |
| `MAX_TOKENS` | Completion length cap. |
| `TEMPERATURE` | Sampling temperature (lower ≈ more deterministic JSON). |
| `HOST` | Bind address (default `0.0.0.0`). |
| `PORT` | Listen port (default **8001**). |
| `LOG_LEVEL` | e.g. `INFO`. |
| `ALLOWED_ORIGINS` | CORS list (parsed as list in settings). |

Example:

```env
GROQ_API_KEY=gsk_...
GROQ_MODEL=llama-3.2-3b-preview
MAX_TOKENS=512
TEMPERATURE=0.7
PORT=8001
```

---

## Install & run

```bash
cd ml_service
python -m venv .venv
# Windows: .venv\Scripts\activate
source .venv/bin/activate
pip install -r requirements.txt
python run.py
```

Service default: **http://127.0.0.1:8001**

For reload during development, set `RELOAD=True` in `.env` or run uvicorn manually:

```bash
uvicorn app.main:app --host 0.0.0.0 --port 8001 --reload
```

### Interactive docs

- **Swagger:** http://localhost:8001/docs  
- **ReDoc:** http://localhost:8001/redoc  

---

## HTTP API (summary)

### `GET /health`

Returns service status and whether the Groq client initialized successfully (see `app/routers/health.py` for exact JSON).

### `POST /api/v1/predict/influencer`

**Body (conceptually):** biography, recent captions/hashtags, follower counts, averages (likes, comments, views), engagement rate, verification flag.

**Response:** `niche_probabilities`, `popularity_probabilities`, `processed_data` (e.g. `top_niche`, `top_popularity`). Exact schema: `app/schemas/influencer.py` and OpenAPI at `/docs`.

### `POST /api/v1/predict/business`

**Body:** `business_description`, `budget_min`, `budget_max`.

**Response:** Same probability shape as influencer prediction; used by Django to seed business–creator matching.

---

## How predictions are produced

1. **Prompts** (`app/utils/prompts.py`) embed profile or business text plus the allowed niche/tier vocabularies.  
2. **Groq** returns natural language; **parser** (`app/utils/parser.py`) extracts JSON-like structures and **normalizes** weights to sum to 1 per distribution where applicable.  
3. **Fallbacks:** on parse failures, code paths typically fall back to **uniform** or safe defaults (see `predictor.py`).

**Niches** and **tiers** are defined in `app/constants.py` and must stay in sync with what the backend and evaluation scripts expect.

---

## Integration with Django

1. Set **`ML_SERVICE_URL`** in `backend/.env` to this service’s base URL.  
2. `MLClient` POSTs to the predict routes and maps responses into influencer/business models and matching inputs.  
3. If the ML service is down, Django may use **fallback** behavior (keyword heuristics or errors, depending on the code path—see backend logs).

**Important:** End-to-end matching **weights** (niche vs budget vs optional audience geography) are implemented in **`backend/businesses/services/matching_service.py`**, not in this repo’s README formulas. Evaluation scripts under `evaluation/` may use documented test weights for offline reports.

---

## Evaluation & offline tests

The **`evaluation/`** directory (when present) may include:

- Dataset CSV paths  
- Scripts such as `run_full_evaluation.py` to batch-call prediction logic and compare rankings  

Run instructions are typically documented in that folder or in the root README. Ensure **the same Groq model** and constants are used when comparing to production behavior.

---

## Troubleshooting

| Symptom | What to check |
|---------|----------------|
| `401` / `model_loaded: false` | `GROQ_API_KEY` missing or invalid |
| `429` | Groq rate limits; reduce concurrency or queue on Django/Celery side |
| Timeouts from Django | `ML_SERVICE_TIMEOUT` in Django settings; Groq latency |
| Odd JSON / empty probabilities | Lower `TEMPERATURE`; inspect raw model output in logs |

---

## Related docs

- [Root README](../README.md)  
- [Backend README](../backend/README.md) — `MLClient`, matching, env vars  
- [Frontend README](../frontend/syncfrontend/README.md)

---

## License

Same as the repository root (proprietary unless otherwise stated).
