<html><body>
<style>

body, h1, h2, h3, div, span, p, pre, a {
  margin: 0;
  padding: 0;
  border: 0;
  font-weight: inherit;
  font-style: inherit;
  font-size: 100%;
  font-family: inherit;
  vertical-align: baseline;
}

body {
  font-size: 13px;
  padding: 1em;
}

h1 {
  font-size: 26px;
  margin-bottom: 1em;
}

h2 {
  font-size: 24px;
  margin-bottom: 1em;
}

h3 {
  font-size: 20px;
  margin-bottom: 1em;
  margin-top: 1em;
}

pre, code {
  line-height: 1.5;
  font-family: Monaco, 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Lucida Console', monospace;
}

pre {
  margin-top: 0.5em;
}

h1, h2, h3, p {
  font-family: Arial, sans serif;
}

h1, h2, h3 {
  border-bottom: solid #CCC 1px;
}

.toc_element {
  margin-top: 0.5em;
}

.firstline {
  margin-left: 2 em;
}

.method  {
  margin-top: 1em;
  border: solid 1px #CCC;
  padding: 1em;
  background: #EEE;
}

.details {
  font-weight: bold;
  font-size: 14px;
}

</style>

<h1><a href="datalabeling_v1beta1.html">Data Labeling API</a> . <a href="datalabeling_v1beta1.projects.html">projects</a> . <a href="datalabeling_v1beta1.projects.datasets.html">datasets</a> . <a href="datalabeling_v1beta1.projects.datasets.evaluations.html">evaluations</a></h1>
<h2>Instance Methods</h2>
<p class="toc_element">
  <code><a href="datalabeling_v1beta1.projects.datasets.evaluations.exampleComparisons.html">exampleComparisons()</a></code>
</p>
<p class="firstline">Returns the exampleComparisons Resource.</p>

<p class="toc_element">
  <code><a href="#close">close()</a></code></p>
<p class="firstline">Close httplib2 connections.</p>
<p class="toc_element">
  <code><a href="#get">get(name, x__xgafv=None)</a></code></p>
<p class="firstline">Gets an evaluation by resource name (to search, use projects.evaluations.search).</p>
<h3>Method Details</h3>
<div class="method">
    <code class="details" id="close">close()</code>
  <pre>Close httplib2 connections.</pre>
</div>

<div class="method">
    <code class="details" id="get">get(name, x__xgafv=None)</code>
  <pre>Gets an evaluation by resource name (to search, use projects.evaluations.search).

Args:
  name: string, Required. Name of the evaluation. Format: &quot;projects/{project_id}/datasets/ {dataset_id}/evaluations/{evaluation_id}&#x27; (required)
  x__xgafv: string, V1 error format.
    Allowed values
      1 - v1 error format
      2 - v2 error format

Returns:
  An object of the form:

    { # Describes an evaluation between a machine learning model&#x27;s predictions and ground truth labels. Created when an EvaluationJob runs successfully.
    &quot;annotationType&quot;: &quot;A String&quot;, # Output only. Type of task that the model version being evaluated performs, as defined in the evaluationJobConfig.inputConfig.annotationType field of the evaluation job that created this evaluation.
    &quot;evaluationMetrics&quot;: { # Output only. Metrics comparing predictions to ground truth labels.
      &quot;classificationMetrics&quot;: { # Metrics calculated for a classification model.
        &quot;prCurve&quot;: { # Precision-recall curve based on ground truth labels, predicted labels, and scores for the predicted labels.
          &quot;meanAveragePrecision&quot;: 3.14, # Mean average prcision of this curve.
          &quot;confidenceMetricsEntries&quot;: [ # Entries that make up the precision-recall graph. Each entry is a &quot;point&quot; on the graph drawn for a different `confidence_threshold`.
            {
              &quot;precisionAt1&quot;: 3.14, # Precision value for entries with label that has highest score.
              &quot;precision&quot;: 3.14, # Precision value.
              &quot;recall&quot;: 3.14, # Recall value.
              &quot;recallAt5&quot;: 3.14, # Recall value for entries with label that has highest 5 scores.
              &quot;recallAt1&quot;: 3.14, # Recall value for entries with label that has highest score.
              &quot;f1ScoreAt5&quot;: 3.14, # The harmonic mean of recall_at5 and precision_at5.
              &quot;f1ScoreAt1&quot;: 3.14, # The harmonic mean of recall_at1 and precision_at1.
              &quot;f1Score&quot;: 3.14, # Harmonic mean of recall and precision.
              &quot;confidenceThreshold&quot;: 3.14, # Threshold used for this entry. For classification tasks, this is a classification threshold: a predicted label is categorized as positive or negative (in the context of this point on the PR curve) based on whether the label&#x27;s score meets this threshold. For image object detection (bounding box) tasks, this is the [intersection-over-union (IOU)](/vision/automl/object-detection/docs/evaluate#intersection-over-union) threshold for the context of this point on the PR curve.
              &quot;precisionAt5&quot;: 3.14, # Precision value for entries with label that has highest 5 scores.
            },
          ],
          &quot;areaUnderCurve&quot;: 3.14, # Area under the precision-recall curve. Not to be confused with area under a receiver operating characteristic (ROC) curve.
          &quot;annotationSpec&quot;: { # Container of information related to one possible annotation that can be used in a labeling task. For example, an image classification task where images are labeled as `dog` or `cat` must reference an AnnotationSpec for `dog` and an AnnotationSpec for `cat`. # The annotation spec of the label for which the precision-recall curve calculated. If this field is empty, that means the precision-recall curve is an aggregate curve for all labels.
            &quot;index&quot;: 42, # Output only. This is the integer index of the AnnotationSpec. The index for the whole AnnotationSpecSet is sequential starting from 0. For example, an AnnotationSpecSet with classes `dog` and `cat`, might contain one AnnotationSpec with `{ display_name: &quot;dog&quot;, index: 0 }` and one AnnotationSpec with `{ display_name: &quot;cat&quot;, index: 1 }`. This is especially useful for model training as it encodes the string labels into numeric values.
            &quot;displayName&quot;: &quot;A String&quot;, # Required. The display name of the AnnotationSpec. Maximum of 64 characters.
            &quot;description&quot;: &quot;A String&quot;, # Optional. User-provided description of the annotation specification. The description can be up to 10,000 characters long.
          },
        },
        &quot;confusionMatrix&quot;: { # Confusion matrix of the model running the classification. Only applicable when the metrics entry aggregates multiple labels. Not applicable when the entry is for a single label. # Confusion matrix of predicted labels vs. ground truth labels.
          &quot;row&quot;: [
            { # A row in the confusion matrix. Each entry in this row has the same ground truth label.
              &quot;annotationSpec&quot;: { # Container of information related to one possible annotation that can be used in a labeling task. For example, an image classification task where images are labeled as `dog` or `cat` must reference an AnnotationSpec for `dog` and an AnnotationSpec for `cat`. # The annotation spec of the ground truth label for this row.
                &quot;index&quot;: 42, # Output only. This is the integer index of the AnnotationSpec. The index for the whole AnnotationSpecSet is sequential starting from 0. For example, an AnnotationSpecSet with classes `dog` and `cat`, might contain one AnnotationSpec with `{ display_name: &quot;dog&quot;, index: 0 }` and one AnnotationSpec with `{ display_name: &quot;cat&quot;, index: 1 }`. This is especially useful for model training as it encodes the string labels into numeric values.
                &quot;displayName&quot;: &quot;A String&quot;, # Required. The display name of the AnnotationSpec. Maximum of 64 characters.
                &quot;description&quot;: &quot;A String&quot;, # Optional. User-provided description of the annotation specification. The description can be up to 10,000 characters long.
              },
              &quot;entries&quot;: [ # A list of the confusion matrix entries. One entry for each possible predicted label.
                {
                  &quot;annotationSpec&quot;: { # Container of information related to one possible annotation that can be used in a labeling task. For example, an image classification task where images are labeled as `dog` or `cat` must reference an AnnotationSpec for `dog` and an AnnotationSpec for `cat`. # The annotation spec of a predicted label.
                    &quot;index&quot;: 42, # Output only. This is the integer index of the AnnotationSpec. The index for the whole AnnotationSpecSet is sequential starting from 0. For example, an AnnotationSpecSet with classes `dog` and `cat`, might contain one AnnotationSpec with `{ display_name: &quot;dog&quot;, index: 0 }` and one AnnotationSpec with `{ display_name: &quot;cat&quot;, index: 1 }`. This is especially useful for model training as it encodes the string labels into numeric values.
                    &quot;displayName&quot;: &quot;A String&quot;, # Required. The display name of the AnnotationSpec. Maximum of 64 characters.
                    &quot;description&quot;: &quot;A String&quot;, # Optional. User-provided description of the annotation specification. The description can be up to 10,000 characters long.
                  },
                  &quot;itemCount&quot;: 42, # Number of items predicted to have this label. (The ground truth label for these items is the `Row.annotationSpec` of this entry&#x27;s parent.)
                },
              ],
            },
          ],
        },
      },
      &quot;objectDetectionMetrics&quot;: { # Metrics calculated for an image object detection (bounding box) model.
        &quot;prCurve&quot;: { # Precision-recall curve.
          &quot;meanAveragePrecision&quot;: 3.14, # Mean average prcision of this curve.
          &quot;confidenceMetricsEntries&quot;: [ # Entries that make up the precision-recall graph. Each entry is a &quot;point&quot; on the graph drawn for a different `confidence_threshold`.
            {
              &quot;precisionAt1&quot;: 3.14, # Precision value for entries with label that has highest score.
              &quot;precision&quot;: 3.14, # Precision value.
              &quot;recall&quot;: 3.14, # Recall value.
              &quot;recallAt5&quot;: 3.14, # Recall value for entries with label that has highest 5 scores.
              &quot;recallAt1&quot;: 3.14, # Recall value for entries with label that has highest score.
              &quot;f1ScoreAt5&quot;: 3.14, # The harmonic mean of recall_at5 and precision_at5.
              &quot;f1ScoreAt1&quot;: 3.14, # The harmonic mean of recall_at1 and precision_at1.
              &quot;f1Score&quot;: 3.14, # Harmonic mean of recall and precision.
              &quot;confidenceThreshold&quot;: 3.14, # Threshold used for this entry. For classification tasks, this is a classification threshold: a predicted label is categorized as positive or negative (in the context of this point on the PR curve) based on whether the label&#x27;s score meets this threshold. For image object detection (bounding box) tasks, this is the [intersection-over-union (IOU)](/vision/automl/object-detection/docs/evaluate#intersection-over-union) threshold for the context of this point on the PR curve.
              &quot;precisionAt5&quot;: 3.14, # Precision value for entries with label that has highest 5 scores.
            },
          ],
          &quot;areaUnderCurve&quot;: 3.14, # Area under the precision-recall curve. Not to be confused with area under a receiver operating characteristic (ROC) curve.
          &quot;annotationSpec&quot;: { # Container of information related to one possible annotation that can be used in a labeling task. For example, an image classification task where images are labeled as `dog` or `cat` must reference an AnnotationSpec for `dog` and an AnnotationSpec for `cat`. # The annotation spec of the label for which the precision-recall curve calculated. If this field is empty, that means the precision-recall curve is an aggregate curve for all labels.
            &quot;index&quot;: 42, # Output only. This is the integer index of the AnnotationSpec. The index for the whole AnnotationSpecSet is sequential starting from 0. For example, an AnnotationSpecSet with classes `dog` and `cat`, might contain one AnnotationSpec with `{ display_name: &quot;dog&quot;, index: 0 }` and one AnnotationSpec with `{ display_name: &quot;cat&quot;, index: 1 }`. This is especially useful for model training as it encodes the string labels into numeric values.
            &quot;displayName&quot;: &quot;A String&quot;, # Required. The display name of the AnnotationSpec. Maximum of 64 characters.
            &quot;description&quot;: &quot;A String&quot;, # Optional. User-provided description of the annotation specification. The description can be up to 10,000 characters long.
          },
        },
      },
    },
    &quot;config&quot;: { # Configuration details used for calculating evaluation metrics and creating an Evaluation. # Output only. Options used in the evaluation job that created this evaluation.
      &quot;boundingBoxEvaluationOptions&quot;: { # Options regarding evaluation between bounding boxes. # Only specify this field if the related model performs image object detection (`IMAGE_BOUNDING_BOX_ANNOTATION`). Describes how to evaluate bounding boxes.
        &quot;iouThreshold&quot;: 3.14, # Minimum [intersection-over-union (IOU)](/vision/automl/object-detection/docs/evaluate#intersection-over-union) required for 2 bounding boxes to be considered a match. This must be a number between 0 and 1.
      },
    },
    &quot;evaluationJobRunTime&quot;: &quot;A String&quot;, # Output only. Timestamp for when the evaluation job that created this evaluation ran.
    &quot;createTime&quot;: &quot;A String&quot;, # Output only. Timestamp for when this evaluation was created.
    &quot;evaluatedItemCount&quot;: &quot;A String&quot;, # Output only. The number of items in the ground truth dataset that were used for this evaluation. Only populated when the evaulation is for certain AnnotationTypes.
    &quot;name&quot;: &quot;A String&quot;, # Output only. Resource name of an evaluation. The name has the following format: &quot;projects/{project_id}/datasets/{dataset_id}/evaluations/ {evaluation_id}&#x27;
  }</pre>
</div>

</body></html>