Source: Deep Learning on Medium
Labeling Guides and Exams: new Supervisely tools to teach workforce on your custom annotation requirements
Creating big and diverse computer vision datasets requires working with a huge labeling workforce. That, in turn, requires professional tools to educate and examine them.
Why do we need Labeling Guides and Labeling Exams? Let’s consider several important observations:
- usually, an annotator has concrete domain specialization: self-driving cars, agriculture, satellite imagery, medicine, and so on
- not every annotator properly understands labeling requirements, thus sometimes they make systematic errors during labeling
- it is a good idea to choose for your custom task the most efficient and accurate labelers based on Quality Score
- from labelers perspective, comprehensive annotation policy should be presented as easy to learn guides and materials
- from time to time company changes annotation policy, all labelers have to be notified and re-examined
Labeling Guides and Labeling Exams in Supervisely are designed to handle all these aspects.