What data annotation covers
Annotation is any task where a person adds the answer a model should learn to give. The data type changes the tools and the skills, but the job is the same: apply a written rule the same way, thousands of times, and flag the cases the rule doesn't cover.
Most projects fall into one of these groups:
- Images and video: bounding boxes, polygons and pixel-level segmentation for computer vision, plus tracking objects from frame to frame.
- Text: sorting messages by topic or intent, tagging names, places, products and amounts, and marking sentiment or tone.
- Audio: transcribing speech, labeling who is speaking and when, and tagging background noise or language switches.
- Search relevance and response rating: judging whether a result answers the query, ranking two or more AI answers, and writing better ones. This is the human feedback used to tune AI assistants, often called RLHF.
- Documents: extracting fields from invoices, forms and contracts, and labeling page layouts so a model learns where things are.
- Content moderation: labeling user posts, images or video against a safety policy, either to moderate live or to train a classifier.
How an annotation project runs
It starts with guidelines. Every label needs a written definition, examples of what counts and what doesn't, and a rule for the hard cases. Vague guidelines are the most common reason labels come back inconsistent, so we spend the first week on them with you.
Then a pilot batch. The team labels a small sample and you review it. The disagreements show where the guidelines are unclear, and fixing them now is far cheaper than relabeling later.
Before anyone works on production data, they pass calibration: labeling a set of gold-standard items where you've already agreed the right answer. Gold items keep appearing in the queue after that, so drift shows up early.
We also track inter-annotator agreement. In plain terms, we give the same items to two or more people and see how often they land on the same label. Low agreement on a label usually means the rule is unclear, not that people are careless, and it goes back into the guidelines at the weekly review.
Security and data handling
The work happens in your systems or the annotation platform you choose, not on our copies of your data. Annotators get named accounts with only the projects they need, and lose access the day they leave the project.
Everyone on the team signs an NDA before they see a single item. Data doesn't leave approved systems: no downloads to personal devices, no screenshots, no outside tools. Where the data includes personal information, we agree with you up front how it's masked, who can see it and what happens to it when the project ends, and we write that into the agreement rather than give you a general assurance.
Managed team or crowdsourcing marketplace
Crowdsourcing marketplaces split your work across many anonymous contributors. They're quick to start and suit simple, high-volume tasks with clear answers, where you can absorb some noise and filter it out with redundancy.
A managed team is the better fit when the task needs judgment, training or context: medical or legal text, long guidelines, response rating, bilingual content, or anything sensitive. The same people stay on your project, learn your edge cases, and can be asked why they labeled something the way they did. You also know who has seen your data.
Many teams use both: a crowd for the easy bulk and a managed team for the hard cases, quality review and gold-standard work.
Content moderation needs a wellbeing plan
Labeling harmful content takes a toll on the people doing it. If your project includes violent, sexual or abusive material, the plan has to account for that from the start, not after someone burns out.
We agree with you how much exposure each person gets per shift, rotate people between harmful and neutral queues, use blurring or grayscale where the tool allows, and agree before the work starts what counseling support and opt-outs from the hardest categories the partner center provides. It's the right way to treat people, and it also keeps judgment steady over months.
Nearshore or offshore for data annotation
Nearshore, in Mexico and Colombia, fits work that changes often and needs fast back-and-forth with your ML team during US hours: new guidelines, pilot batches, response rating where reviewers ask questions as they go. It's also the natural home for Spanish and bilingual English-Spanish data, labeled by native Spanish speakers who know how Latin American users actually write and talk.
Offshore, in the Philippines, India and Egypt, fits large, stable queues with settled guidelines, where work handed off at the end of your day comes back labeled the next morning. It costs less than nearshore, which costs less than onshore. Many projects split the two: guideline work and hard cases nearshore, steady volume offshore.
Questions to ask a data annotation partner
- Ask how quality is measured. You want gold-standard checks, agreement between annotators and a regular review of disagreements, not a single accuracy figure on a slide.
- Ask who the annotators are. Find out whether the same trained people stay on your project or whether work goes out to whoever is online.
- Ask where the data lives. The best answer is your platform, named accounts and no copies outside it.
- Ask what happens when the guidelines change. A good partner retrains, recalibrates and tells you which labels were made under the old rule.
- Ask how moderation work is handled. If the answer doesn't mention exposure limits and support for annotators, look elsewhere.