Case study · Collection pipeline
Damhwa — a community fed by a collection pipeline
Admin dashboard
scheduler, duplicate skip, error logs
Summary + source + link out
how collected material is shown
Live
open it in your browser
Challenge
When a community is fed by collected material, the risk that ends the project is never the layout — it is copyright exposure, load placed on the sources, and a collection run that fails quietly for a week before anyone notices.
Solution
The pipeline was put on screen instead of hidden behind it: a per-source scheduler, counts for collected, new, duplicate, and failed items, response delays, hash-based duplicate skipping, and error logs — with material shown as a summary plus attribution and a link out, never a stored copy, and nothing published until a person approves it.
Results
A build where the operational half of a collection product can be reviewed directly, screen by screen, against the requirements that usually appear as one line in a brief.
- Client
- Self-initiated studio project
- Scope
- Community · collection pipeline · admin
- Stack
- Next.js · Python · scheduler · admin
The real risk sits outside the screen
Anyone commissioning a product that aggregates material from elsewhere is worried about three things, and none of them are visual: a rights dispute, complaints about load on the sources, and the day collection breaks without warning. A demonstration that shows only a beautiful feed answers none of that. So the most carefully built screen here is the admin dashboard, where each source has its own schedule and its own counters, response delays are visible, duplicates are skipped by hash rather than by guesswork, and failures are logged where an operator will actually see them.
Design the exposure, not just the ingestion
How collected material is displayed is a design decision with legal consequences. Nothing is stored as a full copy: each item appears as a summary with clear attribution and a link back to the original, robots rules are respected, collection halts when a source signals it does not want the traffic, and nothing goes live automatically — a person reviews it first. Those four choices lower the exposure structurally, which is a far better position than a disclaimer at the bottom of the page.
Collected and written material in one feed, honestly labelled
The home feed mixes collected items with posts written by members, which only works if a reader can tell them apart at a glance; tags and badges carry that distinction, and the warm cream ground with serif headings keeps the whole thing reading as a community rather than a scraper output. This is a self-initiated studio project built to our own brief and held to production standards, so the feed carries demo content — the pipeline architecture and the console around it are the parts built to be operated.
If your product depends on a pipeline nobody sees, ask us to show you the console first.
sww@celis.co.kr