# MediaFetch Backend — Deployment Guide (Phase 5 + Admin Dashboard)

## Quick Start — Windows (easiest way to run this locally)

1. Install [Node.js](https://nodejs.org) (LTS version) if you haven't already.
2. Double-click **`start_server.bat`** in this folder.
3. First run only: it will automatically run `npm install` and create a
   `.env` file with a default admin login (shown on screen — change the
   password afterward by editing `.env`).
4. Your browser opens automatically to `http://localhost:5000`.
5. Admin dashboard: `http://localhost:5000/admin`

Keep the black terminal window open while using the site — closing it stops
the server. If something goes wrong, this window now stays open and shows
the actual error instead of closing immediately, so you can read what
happened (and paste it back for help, if needed).

## Everything is one deployable unit

This project is now a single Express server that serves all three pieces
from one port:
- `public/index.html` — the public downloader site, served at `/`
- `public/admin.html` — the admin dashboard, served at `/admin`
- Everything under `/api/*` — the backend API both of the above call

Run `npm start` (or `start_server.bat` on Windows) and all three are live
together on `http://localhost:5000`. No separate frontend hosting, no CORS
setup between them — they're same-origin by construction. (If you ever want
to host `index.html` somewhere else instead, see the `API_BASE` comment near
the top of its `<script>` block.)

## ffmpeg works out of the box — no separate install needed

This project bundles ffmpeg via the `ffmpeg-static` npm package, which
downloads a real, working ffmpeg binary for your OS during `npm install`
automatically (Windows, Mac, and Linux all supported). You do **not** need
to separately install ffmpeg and add it to your PATH — `npm install` handles
it. `FFMPEG_PATH` in `.env` is only there if you want to override this with
your own ffmpeg install instead.

## What's in this phase

- `Dockerfile` — production image (Node 20 + ffmpeg + pip-installed yt-dlp)
- `docker-compose.yml` — local/VPS deployment
- `render.yaml` — Render Blueprint (one-click deploy)
- `railway.json` — Railway build/deploy config
- `services/updater.service.js` — daily automatic yt-dlp update + manual admin trigger
- Production hardening in `server.js`: helmet security headers, rate limiting,
  restricted CORS, graceful shutdown, trust proxy
- **Admin Dashboard** (`public/admin.html` at `/admin`) — JWT-authenticated
  UI for managing platforms, ads/announcements, and viewing analytics +
  system health. See the dedicated section below.

---

## Why yt-dlp is installed via pip, not `yt-dlp-exec`'s bundled download

`yt-dlp-exec` normally downloads a standalone binary from GitHub's release API
during `npm install`. That works, but two things make it fragile in
production:

1. GitHub's release API (`api.github.com`) has a **low unauthenticated rate
   limit** — easy to hit on shared build infrastructure or CI runners.
2. There's no clean way to re-trigger that download later without reinstalling
   the npm package.

Instead, the Dockerfile:
1. Installs yt-dlp via `pip3 install yt-dlp` (yt-dlp's own recommended
   distribution channel, hosted on PyPI — no rate limits, always current)
2. Sets `YOUTUBE_DL_SKIP_DOWNLOAD=true` so `yt-dlp-exec`'s postinstall script
   skips its own download attempt
3. Symlinks the pip-installed binary into the exact path `yt-dlp-exec`
   expects (`node_modules/yt-dlp-exec/bin/yt-dlp`)

Your application code (`services/ytdlp.service.js`) needed **zero changes**
for this — it just calls `yt-dlp-exec`, which transparently uses whichever
binary sits at that path. This was verified directly: pip-installing yt-dlp
and symlinking it in, then hitting `/api/health` and `/api/fetch-info`
through the real Phase 1-3 service code, worked without modification.

This is also what makes auto-updates simple: updating yt-dlp is just
`pip3 install -U yt-dlp` — no npm, no GitHub API, no file permissions dance.

---

## FFmpeg configuration

FFmpeg is installed via `apt` (Debian, not Alpine — Alpine's ffmpeg build is
missing codecs commonly needed for cross-platform muxing/audio extraction).
No extra configuration is needed: yt-dlp auto-detects `ffmpeg` on `PATH`. If
you ever need a non-standard location, set `FFMPEG_PATH` in your environment
and pass it to yt-dlp via `--ffmpeg-location` (not currently wired up in the
service code — add it to the `ytdlp()` options in `ytdlp.service.js` if you
need this).

---

## Auto-update configuration for yt-dlp

Two ways to keep yt-dlp current, both already implemented:

### 1. Automatic daily update (on by default)
`services/updater.service.js` runs `pip3 install -U yt-dlp` once ~30 seconds
after server startup, then every `YTDLP_UPDATE_INTERVAL_HOURS` (default 24).
Result is visible in `GET /api/health` under `lastAutoUpdate`.

Disable with `ENABLE_AUTO_UPDATE=false`.

### 2. Manual trigger via the admin dashboard
Log into `/admin` → **System Health** → **Check for Update Now**. This calls
`POST /api/admin/update-ytdlp`, authenticated the same way as the rest of the
admin API (JWT Bearer token from `/api/admin/login` — see the Admin Dashboard
section below). There's no separate API key for this anymore; it was folded
into the main admin auth system for consistency.

---

## Admin Dashboard

A JWT-authenticated dashboard at `/admin` for managing the site without
redeploying code:

- **Platforms** — enable/disable/add/remove the platform resources shown on
  the public site (YouTube, TikTok, Pinterest, etc.)
- **Ads & Announcements** — edit the top banner ad, sidebar ad, and site
  announcement banner HTML/text
- **Overview** — total downloads, downloads by platform, daily volume chart
- **System Health** — yt-dlp/ffmpeg status, Node version, uptime, and a
  manual "check for update" button

### Setup
Set three environment variables (see `.env.example`):
```
ADMIN_USERNAME=admin
ADMIN_PASSWORD=<a strong password>
JWT_SECRET=<a long random string>
```
Generate `JWT_SECRET` with:
```bash
node -e "console.log(require('crypto').randomBytes(48).toString('hex'))"
```
Without these three set, the server still boots (the core downloader API is
unaffected) but `/admin` login and `/api/config` for ads/announcements won't
work correctly — check the startup log for warnings.

### How it works
- **Auth**: `POST /api/admin/login` checks credentials against
  `ADMIN_USERNAME`/`ADMIN_PASSWORD` (timing-safe comparison) and returns a
  JWT (12h expiry). Every other `/api/admin/*` route requires
  `Authorization: Bearer <token>`. Login attempts are rate-limited
  separately (10 per 15 min) to blunt brute-forcing.
- **Storage**: platforms, ad/announcement settings, and the download log
  live in `data/db.json` — a small JSON file, not a database. This is
  intentional for a project this size; see the comment at the top of
  `services/db.service.js` if you outgrow it and want to swap in SQLite/
  Postgres later. Writes are serialized through an internal queue so
  concurrent requests can't corrupt the file.
- **Public config**: the front-end's `index.html` reads `GET /api/config`
  (no auth required) to render only the *enabled* platforms plus the
  current ads/announcement — this is what makes dashboard changes show up
  on the live site immediately, with no code deploy.
- **Analytics**: every `/api/download` request logs `{platform, type,
  success, timestamp}` to `data/db.json`. The platform is detected
  server-side from the URL's hostname (matched against your platform list),
  not trusted from client input, so the numbers can't be gamed by a
  malicious request.

### ⚠️ Persisting `data/db.json` across deploys
This file lives at `/app/data/db.json` inside the container. **If you don't
mount a persistent volume there, every redeploy/restart resets the admin
dashboard to its defaults** (default platform list, no ads, no analytics
history). This is already configured for you:
- Docker Compose: the `mediafetch-data` named volume mounted at `/app/data`
- Render: the blueprint's persistent disk is mounted at `/app/data`
- Railway: add a volume manually at `/app/data` under **Settings → Volumes**
  (Railway's `railway.json` doesn't support declaring volumes in-file)

### Security notes
- The dashboard is a single-admin tool (one username/password pair), not a
  multi-user system — that matches what was asked for, but don't add a
  second admin by just sharing the one password; if you need multiple
  admins with distinct access later, that's a real feature to build, not a
  config tweak.
- Ad/announcement HTML you enter in the dashboard is rendered as-is on the
  public site (`innerHTML`, not escaped) — this is intentional, since ad
  network snippets are scripts/HTML by nature, but it means anyone with
  dashboard access can inject arbitrary HTML/JS into your public site.
  Keep `ADMIN_PASSWORD` and `JWT_SECRET` truly secret.
- `robots: noindex, nofollow` is set on `admin.html` so search engines won't
  index your login page, but that's not a substitute for a strong password —
  the URL itself isn't secret.

---

## Deploying to Render

**Option A — Blueprint (recommended):**
1. Push this repo to GitHub/GitLab.
2. In the Render dashboard: **New → Blueprint**, point it at your repo.
   Render reads `render.yaml` automatically.
3. Set `CORS_ORIGIN`, `ADMIN_USERNAME`, and `ADMIN_PASSWORD` in the
   dashboard (they're marked `sync: false` in the blueprint so they aren't
   checked into git). `JWT_SECRET` is auto-generated for you.
4. Deploy. Render builds the Dockerfile, mounts the persistent disk at
   `/app/data` (admin platforms/settings/analytics — see "Persisting
   data/db.json" above), and health-checks `/api/health`.

**Option B — Manual web service:**
1. **New → Web Service** → connect your repo.
2. Runtime: **Docker** (Render detects the `Dockerfile` automatically).
3. Add environment variables from `.env.example` (at minimum `CORS_ORIGIN`,
   `ADMIN_USERNAME`, `ADMIN_PASSWORD`, `JWT_SECRET`; `PORT` is set
   automatically by Render, don't override it).
4. Health check path: `/api/health`.
5. Under **Disks**, add a 1 GB disk mounted at `/app/data` — this is what
   makes admin-managed platforms/ads/analytics survive redeploys. (`/app/tmp`
   doesn't need a disk; its files are deleted immediately after each
   request, so Render's regular ephemeral filesystem is fine for it.)
6. Choose at least the **Standard** plan — yt-dlp + ffmpeg transcoding is
   CPU/RAM-heavier than a typical Node API, and the free tier will struggle
   or get OOM-killed under real video processing.

---

## Deploying to Railway

1. Push this repo to GitHub.
2. In Railway: **New Project → Deploy from GitHub repo**.
3. Railway auto-detects `railway.json` and builds via the Dockerfile.
4. Under **Variables**, add everything from `.env.example`. Railway sets
   `PORT` automatically — don't override it.
5. Railway's filesystem is ephemeral per-deploy but persists across restarts
   of the same instance. For the admin dashboard's data to survive
   redeploys, add a **Volume** (Settings → Volumes) mounted at `/app/data` —
   this is the one that actually matters; `/app/tmp` is fine ephemeral.
6. Railway health-checks `/api/health` as configured in `railway.json`.

---

## Deploying to a plain VPS (Docker Compose)

```bash
git clone <your-repo>
cd mediafetch-backend
cp .env.example .env
# edit .env: set CORS_ORIGIN, ADMIN_USERNAME, ADMIN_PASSWORD, JWT_SECRET

docker compose up -d --build
```

This builds the image locally and runs it with the settings from `.env`,
persistent named volumes for `/app/tmp` and `/app/data` (the latter is what
keeps admin dashboard data across restarts), and Docker's own healthcheck.
Put this behind a reverse proxy (Caddy/Nginx) for TLS termination — the
app itself only serves plain HTTP on the configured `PORT`.

To update yt-dlp manually on a VPS deployment without redeploying:
```bash
docker compose exec mediafetch-backend pip3 install --break-system-packages -U yt-dlp
```
(The container's built-in daily scheduler does this automatically too.)

---

## Production checklist

- [ ] `CORS_ORIGIN` set to your real front-end domain(s) — never leave as `*`
- [ ] `ADMIN_USERNAME` / `ADMIN_PASSWORD` set to real, strong values (not the
      `.env.example` placeholders)
- [ ] `JWT_SECRET` set to a long random value (see command above)
- [ ] A persistent volume mounted at `/app/data` (Docker Compose and the
      Render blueprint already do this — see the "Persisting data/db.json"
      note above; Railway needs one added manually)
- [ ] `NODE_ENV=production`
- [ ] Plan/instance size large enough for concurrent ffmpeg transcoding
      (video processing is CPU-bound; size for your expected concurrent
      download count, not just idle traffic)
- [ ] Confirm `/api/health` reports `ytdlp.available: true` and
      `ffmpeg.available: true` after first deploy
- [ ] Log into `/admin` once after deploy to confirm login works and
      `data/db.json` is actually persisting (add a test platform, restart
      the container, confirm it's still there)