Labeling Guides + Exams: new Supervisely tools to teach workforce on your custom labeling task

Source: Deep Learning on Medium

Labeling Guides and Exams: new Supervisely tools to teach workforce on your custom annotation requirements

Creating big and diverse computer vision datasets requires working with a huge labeling workforce. That, in turn, requires professional tools to educate and examine them.

Why do we need Labeling Guides and Labeling Exams? Let’s consider several important observations:

  • usually, an annotator has concrete domain specialization: self-driving cars, agriculture, satellite imagery, medicine, and so on
  • not every annotator properly understands labeling requirements, thus sometimes they make systematic errors during labeling
  • it is a good idea to choose for your custom task the most efficient and accurate labelers based on Quality Score
  • from labelers perspective, comprehensive annotation policy should be presented as easy to learn guides and materials
  • from time to time company changes annotation policy, all labelers have to be notified and re-examined

Labeling Guides and Labeling Exams in Supervisely are designed to handle all these aspects.

Labeling Guides —videos, documents, markdown posts are all in one place