threshold values are [0 - 1e-7, 0.5, 1 + 1e-7]

Usage in framework

All in tf.keras.metrics available metrics can be used in our framework. Just use the class name as configuration key and the arguments as as dict.

YAML Configuration

E.g. for tf.metrics.MeanAbsoluteError without no arguments

metrics:
    MeanSquaredError:

or for tf.metrics.MeanIoU with arguments:

metrics:
    MeanIoU:
        num_classes: 7

Keras metrics

AUC class

tensorflow.keras.metrics.AUC(
    num_thresholds=200,
    curve="ROC",
    summation_method="interpolation",
    name=None,
    dtype=None,
    thresholds=None,
    multi_label=False,
    label_weights=None,
)

Computes the approximate AUC (Area under the curve) via a Riemann sum.

This metric creates four local variables, true_positives, true_negatives, false_positives and false_negatives that are used to compute the AUC. To discretize the AUC curve, a linearly spaced set of thresholds is used to compute pairs of recall and precision values. The area under the ROC-curve is therefore computed using the height of the recall values by the false positive rate, while the area under the PR-curve is the computed using the height of the precision values by the recall.

This value is ultimately returned as auc, an idempotent operation that computes the area under a discretized curve of precision versus recall values (computed using the aforementioned variables). The num_thresholds variable controls the degree of discretization with larger numbers of thresholds more closely approximating the true AUC. The quality of the approximation may vary dramatically depending on num_thresholds. The thresholds parameter can be used to manually specify thresholds which split the predictions more evenly.

For best results, predictions should be distributed approximately uniformly in the range [0, 1] and not peaked around 0 or 1. The quality of the AUC approximation may be poor if this is not the case. Setting summation_method to 'minoring' or 'majoring' can help quantify the error in the approximation by providing lower or upper bound estimate of the AUC.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

Args: num_thresholds: (Optional) Defaults to 200. The number of thresholds to use when discretizing the roc curve. Values must be > 1. curve: (Optional) Specifies the name of the curve to be computed, 'ROC' [default] or 'PR' for the Precision-Recall-curve. summation_method: (Optional) Specifies the Riemann summation method used. 'interpolation' (default) applies mid-point summation scheme for ROC. For PR-AUC, interpolates (true/false) positives but not the ratio that is precision (see Davis & Goadrich 2006 for details); 'minoring' applies left summation for increasing intervals and right summation for decreasing intervals; 'majoring' does the opposite. name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result. thresholds: (Optional) A list of floating point values to use as the thresholds for discretizing the curve. If set, the num_thresholds parameter is ignored. Values should be in [0, 1]. Endpoint thresholds equal to {-epsilon, 1+epsilon} for a small positive epsilon value will be automatically included with these to correctly handle predictions equal to exactly 0 or 1. multi_label: boolean indicating whether multilabel data should be treated as such, wherein AUC is computed separately for each label and then averaged across labels, or (when False) if the data should be flattened into a single label before AUC computation. In the latter case, when multilabel data is passed to AUC, each label-prediction pair is treated as an individual data point. Should be set to False for multi-class data. label_weights: (optional) list, array, or tensor of non-negative weights used to compute AUCs for multilabel data. When multi_label is True, the weights are applied to the individual label AUCs when they are averaged to produce the multi-label AUC. When it's False, they are used to weight the individual label predictions in computing the confusion matrix on the flattened data. Note that this is unlike class_weights in that class_weights weights the example depending on the value of its label, whereas label_weights depends only on the index of that label before flattening; therefore label_weights should not be used for multi-class data.

Standalone usage:

m = tf.keras.metrics.AUC(num_thresholds=3) m.update_state([0, 0, 1, 1], [0, 0.5, 0.3, 0.9])

threshold values are [0 - 1e-7, 0.5, 1 + 1e-7]

tp = [2, 1, 0], fp = [2, 0, 0], fn = [0, 1, 2], tn = [0, 2, 2]

recall = [1, 0.5, 0], fp_rate = [1, 0, 0]

auc = ((((1+0.5)/2)(1-0))+ (((0.5+0)/2)(0-0))) = 0.75

m.result().numpy() 0.75

m.reset_states() m.update_state([0, 0, 1, 1], [0, 0.5, 0.3, 0.9], ... sample_weight=[1, 0, 0, 1]) m.result().numpy() 1.0

Usage with compile() API:

model.compile(optimizer='sgd', loss='mse', metrics=[tf.keras.metrics.AUC()])

Accuracy class

tensorflow.keras.metrics.Accuracy(name="accuracy", dtype=None)

Calculates how often predictions equal labels.

This metric creates two local variables, total and count that are used to compute the frequency with which y_pred matches y_true. This frequency is ultimately returned as binary accuracy: an idempotent operation that simply divides total by count.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.Accuracy() m.update_state([[1], [2], [3], [4]], [[0], [2], [3], [4]]) m.result().numpy() 0.75

m.reset_states() m.update_state([[1], [2], [3], [4]], [[0], [2], [3], [4]], ... sample_weight=[1, 1, 0, 0]) m.result().numpy() 0.5

Usage with compile() API:

model.compile(optimizer='sgd',
              loss='mse',
              metrics=[tf.keras.metrics.Accuracy()])

BinaryAccuracy class

tensorflow.keras.metrics.BinaryAccuracy(name="binary_accuracy", dtype=None, threshold=0.5)

Calculates how often predictions match binary labels.

This metric creates two local variables, total and count that are used to compute the frequency with which y_pred matches y_true. This frequency is ultimately returned as binary accuracy: an idempotent operation that simply divides total by count.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result. threshold: (Optional) Float representing the threshold for deciding whether prediction values are 1 or 0.

Standalone usage:

m = tf.keras.metrics.BinaryAccuracy() m.update_state([[1], [1], [0], [0]], [[0.98], [1], [0], [0.6]]) m.result().numpy() 0.75

m.reset_states() m.update_state([[1], [1], [0], [0]], [[0.98], [1], [0], [0.6]], ... sample_weight=[1, 0, 0, 1]) m.result().numpy() 0.5

Usage with compile() API:

model.compile(optimizer='sgd',
              loss='mse',
              metrics=[tf.keras.metrics.BinaryAccuracy()])

BinaryCrossentropy class

tensorflow.keras.metrics.BinaryCrossentropy(
    name="binary_crossentropy", dtype=None, from_logits=False, label_smoothing=0
)

Computes the crossentropy metric between the labels and predictions.

This is the crossentropy metric class to be used when there are only two label classes (0 and 1).

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result. from_logits: (Optional )Whether output is expected to be a logits tensor. By default, we consider that output encodes a probability distribution. label_smoothing: (Optional) Float in [0, 1]. When > 0, label values are smoothed, meaning the confidence on label values are relaxed. e.g. label_smoothing=0.2 means that we will use a value of 0.1 for label 0 and 0.9 for label 1".

Standalone usage:

m = tf.keras.metrics.BinaryCrossentropy() m.update_state([[0, 1], [0, 0]], [[0.6, 0.4], [0.4, 0.6]]) m.result().numpy() 0.81492424

m.reset_states() m.update_state([[0, 1], [0, 0]], [[0.6, 0.4], [0.4, 0.6]], ... sample_weight=[1, 0]) m.result().numpy() 0.9162905

Usage with compile() API:

model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[tf.keras.metrics.BinaryCrossentropy()])

CategoricalAccuracy class

tensorflow.keras.metrics.CategoricalAccuracy(name="categorical_accuracy", dtype=None)

Calculates how often predictions matches one-hot labels.

You can provide logits of classes as y_pred, since argmax of logits and probabilities are same.

This metric creates two local variables, total and count that are used to compute the frequency with which y_pred matches y_true. This frequency is ultimately returned as categorical accuracy: an idempotent operation that simply divides total by count.

y_pred and y_true should be passed in as vectors of probabilities, rather than as labels. If necessary, use tf.one_hot to expand y_true as a vector.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.CategoricalAccuracy() m.update_state([[0, 0, 1], [0, 1, 0]], [[0.1, 0.9, 0.8], ... [0.05, 0.95, 0]]) m.result().numpy() 0.5

m.reset_states() m.update_state([[0, 0, 1], [0, 1, 0]], [[0.1, 0.9, 0.8], ... [0.05, 0.95, 0]], ... sample_weight=[0.7, 0.3]) m.result().numpy() 0.3

Usage with compile() API:

model.compile(
  optimizer='sgd',
  loss='mse',
  metrics=[tf.keras.metrics.CategoricalAccuracy()])

CategoricalCrossentropy class

tensorflow.keras.metrics.CategoricalCrossentropy(
    name="categorical_crossentropy", dtype=None, from_logits=False, label_smoothing=0
)

Computes the crossentropy metric between the labels and predictions.

This is the crossentropy metric class to be used when there are multiple label classes (2 or more). Here we assume that labels are given as a one_hot representation. eg., When labels values are [2, 0, 1], y_true = [[0, 0, 1], [1, 0, 0], [0, 1, 0]].

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result. from_logits: (Optional) Whether output is expected to be a logits tensor. By default, we consider that output encodes a probability distribution. label_smoothing: (Optional) Float in [0, 1]. When > 0, label values are smoothed, meaning the confidence on label values are relaxed. e.g. label_smoothing=0.2 means that we will use a value of 0.1 for label 0 and 0.9 for label 1"

Standalone usage:

EPSILON = 1e-7, y = y_true, y` = y_pred

y` = clip_ops.clip_by_value(output, EPSILON, 1. - EPSILON)

y` = [[0.05, 0.95, EPSILON], [0.1, 0.8, 0.1]]

xent = -sum(y * log(y'), axis = -1)

= -((log 0.95), (log 0.1))

= [0.051, 2.302]

Reduced xent = (0.051 + 2.302) / 2

m = tf.keras.metrics.CategoricalCrossentropy() m.update_state([[0, 1, 0], [0, 0, 1]], ... [[0.05, 0.95, 0], [0.1, 0.8, 0.1]]) m.result().numpy() 1.1769392

m.reset_states() m.update_state([[0, 1, 0], [0, 0, 1]], ... [[0.05, 0.95, 0], [0.1, 0.8, 0.1]], ... sample_weight=tf.constant([0.3, 0.7])) m.result().numpy() 1.6271976

Usage with compile() API:

model.compile(
  optimizer='sgd',
  loss='mse',
  metrics=[tf.keras.metrics.CategoricalCrossentropy()])

CategoricalHinge class

tensorflow.keras.metrics.CategoricalHinge(name="categorical_hinge", dtype=None)

Computes the categorical hinge metric between y_true and y_pred.

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.CategoricalHinge() m.update_state([[0, 1], [0, 0]], [[0.6, 0.4], [0.4, 0.6]]) m.result().numpy() 1.4000001

m.reset_states() m.update_state([[0, 1], [0, 0]], [[0.6, 0.4], [0.4, 0.6]], ... sample_weight=[1, 0]) m.result().numpy() 1.2

Usage with compile() API:

model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[tf.keras.metrics.CategoricalHinge()])

CosineSimilarity class

tensorflow.keras.metrics.CosineSimilarity(name="cosine_similarity", dtype=None, axis=-1)

Computes the cosine similarity between the labels and predictions.

cosine similarity = (a . b) / ||a|| ||b||

See: Cosine Similarity.

This metric keeps the average cosine similarity between predictions and labels over a stream of data.

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result. axis: (Optional) Defaults to -1. The dimension along which the cosine similarity is computed.

Standalone usage:

l2_norm(y_true) = [[0., 1.], [1./1.414], 1./1.414]]]

l2_norm(y_pred) = [[1., 0.], [1./1.414], 1./1.414]]]

l2_norm(y_true) . l2_norm(y_pred) = [[0., 0.], [0.5, 0.5]]

result = mean(sum(l2_norm(y_true) . l2_norm(y_pred), axis=1))

= ((0. + 0.) + (0.5 + 0.5)) / 2

m = tf.keras.metrics.CosineSimilarity(axis=1) m.update_state([[0., 1.], [1., 1.]], [[1., 0.], [1., 1.]]) m.result().numpy() 0.49999997

m.reset_states() m.update_state([[0., 1.], [1., 1.]], [[1., 0.], [1., 1.]], ... sample_weight=[0.3, 0.7]) m.result().numpy() 0.6999999

Usage with compile() API:

model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[tf.keras.metrics.CosineSimilarity(axis=1)])

FalseNegatives class

tensorflow.keras.metrics.FalseNegatives(thresholds=None, name=None, dtype=None)

Calculates the number of false negatives.

If sample_weight is given, calculates the sum of the weights of false negatives. This metric creates one local variable, accumulator that is used to keep track of the number of false negatives.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

Args: thresholds: (Optional) Defaults to 0.5. A float value or a python list/tuple of float threshold values in [0, 1]. A threshold is compared with prediction values to determine the truth value of predictions (i.e., above the threshold is true, below is false). One metric value is generated for each threshold value. name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.FalseNegatives() m.update_state([0, 1, 1, 1], [0, 1, 0, 0]) m.result().numpy() 2.0

m.reset_states() m.update_state([0, 1, 1, 1], [0, 1, 0, 0], sample_weight=[0, 0, 1, 0]) m.result().numpy() 1.0

Usage with compile() API:

model.compile(optimizer='sgd',
              loss='mse',
              metrics=[tf.keras.metrics.FalseNegatives()])

FalsePositives class

tensorflow.keras.metrics.FalsePositives(thresholds=None, name=None, dtype=None)

Calculates the number of false positives.

If sample_weight is given, calculates the sum of the weights of false positives. This metric creates one local variable, accumulator that is used to keep track of the number of false positives.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

Args: thresholds: (Optional) Defaults to 0.5. A float value or a python list/tuple of float threshold values in [0, 1]. A threshold is compared with prediction values to determine the truth value of predictions (i.e., above the threshold is true, below is false). One metric value is generated for each threshold value. name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.FalsePositives() m.update_state([0, 1, 0, 0], [0, 0, 1, 1]) m.result().numpy() 2.0

m.reset_states() m.update_state([0, 1, 0, 0], [0, 0, 1, 1], sample_weight=[0, 0, 1, 0]) m.result().numpy() 1.0

Usage with compile() API:

model.compile(optimizer='sgd',
              loss='mse',
              metrics=[tf.keras.metrics.FalsePositives()])

Hinge class

tensorflow.keras.metrics.Hinge(name="hinge", dtype=None)

Computes the hinge metric between y_true and y_pred.

y_true values are expected to be -1 or 1. If binary (0 or 1) labels are provided we will convert them to -1 or 1.

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.Hinge() m.update_state([[0, 1], [0, 0]], [[0.6, 0.4], [0.4, 0.6]]) m.result().numpy() 1.3

m.reset_states() m.update_state([[0, 1], [0, 0]], [[0.6, 0.4], [0.4, 0.6]], ... sample_weight=[1, 0]) m.result().numpy() 1.1

Usage with compile() API:

model.compile(optimizer='sgd', loss='mse', metrics=[tf.keras.metrics.Hinge()])

kl_divergence function

tensorflow.keras.metrics.KLD(y_true, y_pred)

Computes Kullback-Leibler divergence loss between y_true and y_pred.

loss = y_true * log(y_true / y_pred)

See: https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence

Standalone usage:

y_true = np.random.randint(0, 2, size=(2, 3)).astype(np.float64) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.kullback_leibler_divergence(y_true, y_pred) assert loss.shape == (2,) y_true = tf.keras.backend.clip(y_true, 1e-7, 1) y_pred = tf.keras.backend.clip(y_pred, 1e-7, 1) assert np.array_equal( ... loss.numpy(), np.sum(y_true * np.log(y_true / y_pred), axis=-1))

Args: y_true: Tensor of true targets. y_pred: Tensor of predicted targets.

Returns: A Tensor with loss.

Raises: TypeError: If y_true cannot be cast to the y_pred.dtype.

KLDivergence class

tensorflow.keras.metrics.KLDivergence(name="kullback_leibler_divergence", dtype=None)

Computes Kullback-Leibler divergence metric between y_true and y_pred.

metric = y_true * log(y_true / y_pred)

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.KLDivergence() m.update_state([[0, 1], [0, 0]], [[0.6, 0.4], [0.4, 0.6]]) m.result().numpy() 0.45814306

m.reset_states() m.update_state([[0, 1], [0, 0]], [[0.6, 0.4], [0.4, 0.6]], ... sample_weight=[1, 0]) m.result().numpy() 0.9162892

Usage with compile() API:

model.compile(optimizer='sgd',
              loss='mse',
              metrics=[tf.keras.metrics.KLDivergence()])

LogCoshError class

tensorflow.keras.metrics.LogCoshError(name="logcosh", dtype=None)

Computes the logarithm of the hyperbolic cosine of the prediction error.

logcosh = log((exp(x) + exp(-x))/2), where x is the error (y_pred - y_true)

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.LogCoshError() m.update_state([[0, 1], [0, 0]], [[1, 1], [0, 0]]) m.result().numpy() 0.10844523

m.reset_states() m.update_state([[0, 1], [0, 0]], [[1, 1], [0, 0]], ... sample_weight=[1, 0]) m.result().numpy() 0.21689045

Usage with compile() API:

model.compile(optimizer='sgd',
              loss='mse',
              metrics=[tf.keras.metrics.LogCoshError()])

mean_absolute_error function

tensorflow.keras.metrics.MAE(y_true, y_pred)

Computes the mean absolute error between labels and predictions.

loss = mean(abs(y_true - y_pred), axis=-1)

Standalone usage:

y_true = np.random.randint(0, 2, size=(2, 3)) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.mean_absolute_error(y_true, y_pred) assert loss.shape == (2,) assert np.array_equal( ... loss.numpy(), np.mean(np.abs(y_true - y_pred), axis=-1))

Args: y_true: Ground truth values. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN].

Returns: Mean absolute error values. shape = [batch_size, d0, .. dN-1].

mean_absolute_percentage_error function

tensorflow.keras.metrics.MAPE(y_true, y_pred)

Computes the mean absolute percentage error between y_true and y_pred.

loss = 100 * mean(abs((y_true - y_pred) / y_true), axis=-1)

Standalone usage:

y_true = np.random.random(size=(2, 3)) y_true = np.maximum(y_true, 1e-7) # Prevent division by zero y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.mean_absolute_percentage_error(y_true, y_pred) assert loss.shape == (2,) assert np.array_equal( ... loss.numpy(), ... 100. * np.mean(np.abs((y_true - y_pred) / y_true), axis=-1))

Args: y_true: Ground truth values. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN].

Returns: Mean absolute percentage error values. shape = [batch_size, d0, .. dN-1].

mean_squared_error function

tensorflow.keras.metrics.MSE(y_true, y_pred)

Computes the mean squared error between labels and predictions.

After computing the squared distance between the inputs, the mean value over the last dimension is returned.

loss = mean(square(y_true - y_pred), axis=-1)

Standalone usage:

y_true = np.random.randint(0, 2, size=(2, 3)) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.mean_squared_error(y_true, y_pred) assert loss.shape == (2,) assert np.array_equal( ... loss.numpy(), np.mean(np.square(y_true - y_pred), axis=-1))

Args: y_true: Ground truth values. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN].

Returns: Mean squared error values. shape = [batch_size, d0, .. dN-1].

mean_squared_logarithmic_error function

tensorflow.keras.metrics.MSLE(y_true, y_pred)

Computes the mean squared logarithmic error between y_true and y_pred.

loss = mean(square(log(y_true + 1) - log(y_pred + 1)), axis=-1)

Standalone usage:

y_true = np.random.randint(0, 2, size=(2, 3)) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.mean_squared_logarithmic_error(y_true, y_pred) assert loss.shape == (2,) y_true = np.maximum(y_true, 1e-7) y_pred = np.maximum(y_pred, 1e-7) assert np.allclose( ... loss.numpy(), ... np.mean( ... np.square(np.log(y_true + 1.) - np.log(y_pred + 1.)), axis=-1))

Args: y_true: Ground truth values. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN].

Returns: Mean squared logarithmic error values. shape = [batch_size, d0, .. dN-1].

Mean class

tensorflow.keras.metrics.Mean(name="mean", dtype=None)

Computes the (weighted) mean of the given values.

For example, if values is [1, 3, 5, 7] then the mean is 4. If the weights were specified as [1, 1, 0, 0] then the mean would be 2.

This metric creates two variables, total and count that are used to compute the average of values. This average is ultimately returned as mean which is an idempotent operation that simply divides total by count.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.Mean() m.update_state([1, 3, 5, 7]) m.result().numpy() 4.0 m.reset_states() m.update_state([1, 3, 5, 7], sample_weight=[1, 1, 0, 0]) m.result().numpy() 2.0

Usage with compile() API:

model.add_metric(tf.keras.metrics.Mean(name='mean_1')(outputs))
model.compile(optimizer='sgd', loss='mse')

MeanAbsoluteError class

tensorflow.keras.metrics.MeanAbsoluteError(name="mean_absolute_error", dtype=None)

Computes the mean absolute error between the labels and predictions.

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.MeanAbsoluteError() m.update_state([[0, 1], [0, 0]], [[1, 1], [0, 0]]) m.result().numpy() 0.25

m.reset_states() m.update_state([[0, 1], [0, 0]], [[1, 1], [0, 0]], ... sample_weight=[1, 0]) m.result().numpy() 0.5

Usage with compile() API:

model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[tf.keras.metrics.MeanAbsoluteError()])

MeanAbsolutePercentageError class

tensorflow.keras.metrics.MeanAbsolutePercentageError(name="mean_absolute_percentage_error", dtype=None)

Computes the mean absolute percentage error between y_true and y_pred.

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.MeanAbsolutePercentageError() m.update_state([[0, 1], [0, 0]], [[1, 1], [0, 0]]) m.result().numpy() 250000000.0

m.reset_states() m.update_state([[0, 1], [0, 0]], [[1, 1], [0, 0]], ... sample_weight=[1, 0]) m.result().numpy() 500000000.0

Usage with compile() API:

model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[tf.keras.metrics.MeanAbsolutePercentageError()])

MeanIoU class

tensorflow.keras.metrics.MeanIoU(num_classes, name=None, dtype=None)

Computes the mean Intersection-Over-Union metric.

Mean Intersection-Over-Union is a common evaluation metric for semantic image segmentation, which first computes the IOU for each semantic class and then computes the average over classes. IOU is defined as follows: IOU = true_positive / (true_positive + false_positive + false_negative). The predictions are accumulated in a confusion matrix, weighted by sample_weight and the metric is then calculated from it.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

Args: num_classes: The possible number of labels the prediction task can have. This value must be provided, since a confusion matrix of dimension = [num_classes, num_classes] will be allocated. name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

cm = [[1, 1],

[1, 1]]

sum_row = [2, 2], sum_col = [2, 2], true_positives = [1, 1]

iou = true_positives / (sum_row + sum_col - true_positives))

result = (1 / (2 + 2 - 1) + 1 / (2 + 2 - 1)) / 2 = 0.33

m = tf.keras.metrics.MeanIoU(num_classes=2) m.update_state([0, 0, 1, 1], [0, 1, 0, 1]) m.result().numpy() 0.33333334

m.reset_states() m.update_state([0, 0, 1, 1], [0, 1, 0, 1], ... sample_weight=[0.3, 0.3, 0.3, 0.1]) m.result().numpy() 0.23809525

Usage with compile() API:

model.compile(
  optimizer='sgd',
  loss='mse',
  metrics=[tf.keras.metrics.MeanIoU(num_classes=2)])

MeanRelativeError class

tensorflow.keras.metrics.MeanRelativeError(normalizer, name=None, dtype=None)

Computes the mean relative error by normalizing with the given values.

This metric creates two local variables, total and count that are used to compute the mean relative error. This is weighted by sample_weight, and it is ultimately returned as mean_relative_error: an idempotent operation that simply divides total by count.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

Args: normalizer: The normalizer values with same shape as predictions. name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.MeanRelativeError(normalizer=[1, 3, 2, 3]) m.update_state([1, 3, 2, 3], [2, 4, 6, 8])

metric = mean(|y_pred - y_true| / normalizer)

= mean([1, 1, 4, 5] / [1, 3, 2, 3]) = mean([1, 1/3, 2, 5/3])

= 5/4 = 1.25

m.result().numpy() 1.25

Usage with compile() API:

model.compile(
  optimizer='sgd',
  loss='mse',
  metrics=[tf.keras.metrics.MeanRelativeError(normalizer=[1, 3])])

MeanSquaredError class

tensorflow.keras.metrics.MeanSquaredError(name="mean_squared_error", dtype=None)

Computes the mean squared error between y_true and y_pred.

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.MeanSquaredError() m.update_state([[0, 1], [0, 0]], [[1, 1], [0, 0]]) m.result().numpy() 0.25

m.reset_states() m.update_state([[0, 1], [0, 0]], [[1, 1], [0, 0]], ... sample_weight=[1, 0]) m.result().numpy() 0.5

Usage with compile() API:

model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[tf.keras.metrics.MeanSquaredError()])

MeanSquaredLogarithmicError class

tensorflow.keras.metrics.MeanSquaredLogarithmicError(name="mean_squared_logarithmic_error", dtype=None)

Computes the mean squared logarithmic error between y_true and y_pred.

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.MeanSquaredLogarithmicError() m.update_state([[0, 1], [0, 0]], [[1, 1], [0, 0]]) m.result().numpy() 0.12011322

m.reset_states() m.update_state([[0, 1], [0, 0]], [[1, 1], [0, 0]], ... sample_weight=[1, 0]) m.result().numpy() 0.24022643

Usage with compile() API:

model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[tf.keras.metrics.MeanSquaredLogarithmicError()])

MeanTensor class

tensorflow.keras.metrics.MeanTensor(name="mean_tensor", dtype=None)

Computes the element-wise (weighted) mean of the given tensors.

MeanTensor returns a tensor with the same shape of the input tensors. The mean value is updated by keeping local variables total and count. The total tracks the sum of the weighted values, and count stores the sum of the weighted counts.

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.MeanTensor() m.update_state([0, 1, 2, 3]) m.update_state([4, 5, 6, 7]) m.result().numpy() array([2., 3., 4., 5.], dtype=float32)

m.update_state([12, 10, 8, 6], sample_weight= [0, 0.2, 0.5, 1]) m.result().numpy() array([2. , 3.6363635, 4.8 , 5.3333335], dtype=float32)

Metric class

tensorflow.keras.metrics.Metric(name=None, dtype=None, **kwargs)

Encapsulates metric logic and state.

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result. **kwargs: Additional layer keywords arguments.

Standalone usage:

m = SomeMetric(...)
for input in ...:
  m.update_state(input)
print('Final result: ', m.result().numpy())

Usage with compile() API:

model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(64, activation='relu'))
model.add(tf.keras.layers.Dense(64, activation='relu'))
model.add(tf.keras.layers.Dense(10, activation='softmax'))

model.compile(optimizer=tf.keras.optimizers.RMSprop(0.01),
              loss=tf.keras.losses.CategoricalCrossentropy(),
              metrics=[tf.keras.metrics.CategoricalAccuracy()])

data = np.random.random((1000, 32))
labels = np.random.random((1000, 10))

dataset = tf.data.Dataset.from_tensor_slices((data, labels))
dataset = dataset.batch(32)

model.fit(dataset, epochs=10)

To be implemented by subclasses: * __init__(): All state variables should be created in this method by calling self.add_weight() like: self.var = self.add_weight(...) * update_state(): Has all updates to the state variables like: self.var.assign_add(...). * result(): Computes and returns a value for the metric from the state variables.

Example subclass implementation:

class BinaryTruePositives(tf.keras.metrics.Metric):

  def __init__(self, name='binary_true_positives', **kwargs):
    super(BinaryTruePositives, self).__init__(name=name, **kwargs)
    self.true_positives = self.add_weight(name='tp', initializer='zeros')

  def update_state(self, y_true, y_pred, sample_weight=None):
    y_true = tf.cast(y_true, tf.bool)
    y_pred = tf.cast(y_pred, tf.bool)

    values = tf.logical_and(tf.equal(y_true, True), tf.equal(y_pred, True))
    values = tf.cast(values, self.dtype)
    if sample_weight is not None:
      sample_weight = tf.cast(sample_weight, self.dtype)
      sample_weight = tf.broadcast_to(sample_weight, values.shape)
      values = tf.multiply(values, sample_weight)
    self.true_positives.assign_add(tf.reduce_sum(values))

  def result(self):
    return self.true_positives

Poisson class

tensorflow.keras.metrics.Poisson(name="poisson", dtype=None)

Computes the Poisson metric between y_true and y_pred.

metric = y_pred - y_true * log(y_pred)

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.Poisson() m.update_state([[0, 1], [0, 0]], [[1, 1], [0, 0]]) m.result().numpy() 0.49999997

m.reset_states() m.update_state([[0, 1], [0, 0]], [[1, 1], [0, 0]], ... sample_weight=[1, 0]) m.result().numpy() 0.99999994

Usage with compile() API:

model.compile(optimizer='sgd',
              loss='mse',
              metrics=[tf.keras.metrics.Poisson()])

Precision class

tensorflow.keras.metrics.Precision(thresholds=None, top_k=None, class_id=None, name=None, dtype=None)

Computes the precision of the predictions with respect to the labels.

The metric creates two local variables, true_positives and false_positives that are used to compute the precision. This value is ultimately returned as precision, an idempotent operation that simply divides true_positives by the sum of true_positives and false_positives.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

If top_k is set, we'll calculate precision as how often on average a class among the top-k classes with the highest predicted values of a batch entry is correct and can be found in the label for that entry.

If class_id is specified, we calculate precision by considering only the entries in the batch for which class_id is above the threshold and/or in the top-k highest predictions, and computing the fraction of them for which class_id is indeed a correct label.

Args: thresholds: (Optional) A float value or a python list/tuple of float threshold values in [0, 1]. A threshold is compared with prediction values to determine the truth value of predictions (i.e., above the threshold is true, below is false). One metric value is generated for each threshold value. If neither thresholds nor top_k are set, the default is to calculate precision with thresholds=0.5. top_k: (Optional) Unset by default. An int value specifying the top-k predictions to consider when calculating precision. class_id: (Optional) Integer class ID for which we want binary metrics. This must be in the half-open interval [0, num_classes), where num_classes is the last dimension of predictions. name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.Precision() m.update_state([0, 1, 1, 1], [1, 0, 1, 1]) m.result().numpy() 0.6666667

m.reset_states() m.update_state([0, 1, 1, 1], [1, 0, 1, 1], sample_weight=[0, 0, 1, 0]) m.result().numpy() 1.0

With top_k=2, it will calculate precision over y_true[:2] and y_pred[:2]

m = tf.keras.metrics.Precision(top_k=2) m.update_state([0, 0, 1, 1], [1, 1, 1, 1]) m.result().numpy() 0.0

With top_k=4, it will calculate precision over y_true[:4] and y_pred[:4]

m = tf.keras.metrics.Precision(top_k=4) m.update_state([0, 0, 1, 1], [1, 1, 1, 1]) m.result().numpy() 0.5

Usage with compile() API:

model.compile(optimizer='sgd',
              loss='mse',
              metrics=[tf.keras.metrics.Precision()])

PrecisionAtRecall class

tensorflow.keras.metrics.PrecisionAtRecall(recall, num_thresholds=200, name=None, dtype=None)

Computes best precision where recall is >= specified value.

This metric creates four local variables, true_positives, true_negatives, false_positives and false_negatives that are used to compute the precision at the given recall. The threshold for the given recall value is computed and used to evaluate the corresponding precision.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

Args: recall: A scalar value in range [0, 1]. num_thresholds: (Optional) Defaults to 200. The number of thresholds to use for matching the given recall. name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.PrecisionAtRecall(0.5) m.update_state([0, 0, 0, 1, 1], [0, 0.3, 0.8, 0.3, 0.8]) m.result().numpy() 0.5

m.reset_states() m.update_state([0, 0, 0, 1, 1], [0, 0.3, 0.8, 0.3, 0.8], ... sample_weight=[2, 2, 2, 1, 1]) m.result().numpy() 0.33333333

Usage with compile() API:

model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[tf.keras.metrics.PrecisionAtRecall(recall=0.8)])

Recall class

tensorflow.keras.metrics.Recall(thresholds=None, top_k=None, class_id=None, name=None, dtype=None)

Computes the recall of the predictions with respect to the labels.

This metric creates two local variables, true_positives and false_negatives, that are used to compute the recall. This value is ultimately returned as recall, an idempotent operation that simply divides true_positives by the sum of true_positives and false_negatives.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

If top_k is set, recall will be computed as how often on average a class among the labels of a batch entry is in the top-k predictions.

If class_id is specified, we calculate recall by considering only the entries in the batch for which class_id is in the label, and computing the fraction of them for which class_id is above the threshold and/or in the top-k predictions.

Args: thresholds: (Optional) A float value or a python list/tuple of float threshold values in [0, 1]. A threshold is compared with prediction values to determine the truth value of predictions (i.e., above the threshold is true, below is false). One metric value is generated for each threshold value. If neither thresholds nor top_k are set, the default is to calculate recall with thresholds=0.5. top_k: (Optional) Unset by default. An int value specifying the top-k predictions to consider when calculating recall. class_id: (Optional) Integer class ID for which we want binary metrics. This must be in the half-open interval [0, num_classes), where num_classes is the last dimension of predictions. name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.Recall() m.update_state([0, 1, 1, 1], [1, 0, 1, 1]) m.result().numpy() 0.6666667

m.reset_states() m.update_state([0, 1, 1, 1], [1, 0, 1, 1], sample_weight=[0, 0, 1, 0]) m.result().numpy() 1.0

Usage with compile() API:

model.compile(optimizer='sgd',
              loss='mse',
              metrics=[tf.keras.metrics.Recall()])

RecallAtPrecision class

tensorflow.keras.metrics.RecallAtPrecision(precision, num_thresholds=200, name=None, dtype=None)

Computes best recall where precision is >= specified value.

For a given score-label-distribution the required precision might not be achievable, in this case 0.0 is returned as recall.

This metric creates four local variables, true_positives, true_negatives, false_positives and false_negatives that are used to compute the recall at the given precision. The threshold for the given precision value is computed and used to evaluate the corresponding recall.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

Args: precision: A scalar value in range [0, 1]. num_thresholds: (Optional) Defaults to 200. The number of thresholds to use for matching the given precision. name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.RecallAtPrecision(0.8) m.update_state([0, 0, 1, 1], [0, 0.5, 0.3, 0.9]) m.result().numpy() 0.5

m.reset_states() m.update_state([0, 0, 1, 1], [0, 0.5, 0.3, 0.9], ... sample_weight=[1, 0, 0, 1]) m.result().numpy() 1.0

Usage with compile() API:

model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[tf.keras.metrics.RecallAtPrecision(precision=0.8)])

RootMeanSquaredError class

tensorflow.keras.metrics.RootMeanSquaredError(name="root_mean_squared_error", dtype=None)

Computes root mean squared error metric between y_true and y_pred.

Standalone usage:

m = tf.keras.metrics.RootMeanSquaredError() m.update_state([[0, 1], [0, 0]], [[1, 1], [0, 0]]) m.result().numpy() 0.5

m.reset_states() m.update_state([[0, 1], [0, 0]], [[1, 1], [0, 0]], ... sample_weight=[1, 0]) m.result().numpy() 0.70710677

Usage with compile() API:

model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[tf.keras.metrics.RootMeanSquaredError()])

SensitivityAtSpecificity class

tensorflow.keras.metrics.SensitivityAtSpecificity(specificity, num_thresholds=200, name=None, dtype=None)

Computes best sensitivity where specificity is >= specified value.

the sensitivity at a given specificity.

Sensitivity measures the proportion of actual positives that are correctly identified as such (tp / (tp + fn)). Specificity measures the proportion of actual negatives that are correctly identified as such (tn / (tn + fp)).

This metric creates four local variables, true_positives, true_negatives, false_positives and false_negatives that are used to compute the sensitivity at the given specificity. The threshold for the given specificity value is computed and used to evaluate the corresponding sensitivity.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

For additional information about specificity and sensitivity, see the following.

Args: specificity: A scalar value in range [0, 1]. num_thresholds: (Optional) Defaults to 200. The number of thresholds to use for matching the given specificity. name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.SensitivityAtSpecificity(0.5) m.update_state([0, 0, 0, 1, 1], [0, 0.3, 0.8, 0.3, 0.8]) m.result().numpy() 0.5

m.reset_states() m.update_state([0, 0, 0, 1, 1], [0, 0.3, 0.8, 0.3, 0.8], ... sample_weight=[1, 1, 2, 2, 1]) m.result().numpy() 0.333333

Usage with compile() API:

model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[tf.keras.metrics.SensitivityAtSpecificity()])

SparseCategoricalAccuracy class

tensorflow.keras.metrics.SparseCategoricalAccuracy(name="sparse_categorical_accuracy", dtype=None)

Calculates how often predictions matches integer labels.

acc = np.dot(sample_weight, np.equal(y_true, np.argmax(y_pred, axis=1))

You can provide logits of classes as y_pred, since argmax of logits and probabilities are same.

This metric creates two local variables, total and count that are used to compute the frequency with which y_pred matches y_true. This frequency is ultimately returned as sparse categorical accuracy: an idempotent operation that simply divides total by count.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.SparseCategoricalAccuracy() m.update_state([[2], [1]], [[0.1, 0.6, 0.3], [0.05, 0.95, 0]]) m.result().numpy() 0.5

m.reset_states() m.update_state([[2], [1]], [[0.1, 0.6, 0.3], [0.05, 0.95, 0]], ... sample_weight=[0.7, 0.3]) m.result().numpy() 0.3

Usage with compile() API:

model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[tf.keras.metrics.SparseCategoricalAccuracy()])

SparseCategoricalCrossentropy class

tensorflow.keras.metrics.SparseCategoricalCrossentropy(
    name="sparse_categorical_crossentropy", dtype=None, from_logits=False, axis=-1
)

Computes the crossentropy metric between the labels and predictions.

Use this crossentropy metric when there are two or more label classes. We expect labels to be provided as integers. If you want to provide labels using one-hot representation, please use CategoricalCrossentropy metric. There should be # classes floating point values per feature for y_pred and a single floating point value per feature for y_true.

In the snippet below, there is a single floating point value per example for y_true and # classes floating pointing values per example for y_pred. The shape of y_true is [batch_size] and the shape of y_pred is [batch_size, num_classes].

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result. from_logits: (Optional) Whether output is expected to be a logits tensor. By default, we consider that output encodes a probability distribution. axis: (Optional) Defaults to -1. The dimension along which the metric is computed.

Standalone usage:

y_true = one_hot(y_true) = [[0, 1, 0], [0, 0, 1]]

logits = log(y_pred)

softmax = exp(logits) / sum(exp(logits), axis=-1)

softmax = [[0.05, 0.95, EPSILON], [0.1, 0.8, 0.1]]

xent = -sum(y * log(softmax), 1)

log(softmax) = [[-2.9957, -0.0513, -16.1181],

[-2.3026, -0.2231, -2.3026]]

y_true * log(softmax) = [[0, -0.0513, 0], [0, 0, -2.3026]]

xent = [0.0513, 2.3026]

Reduced xent = (0.0513 + 2.3026) / 2

m = tf.keras.metrics.SparseCategoricalCrossentropy() m.update_state([1, 2], ... [[0.05, 0.95, 0], [0.1, 0.8, 0.1]]) m.result().numpy() 1.1769392

m.reset_states() m.update_state([1, 2], ... [[0.05, 0.95, 0], [0.1, 0.8, 0.1]], ... sample_weight=tf.constant([0.3, 0.7])) m.result().numpy() 1.6271976

Usage with compile() API:

model.compile(
  optimizer='sgd',
  loss='mse',
  metrics=[tf.keras.metrics.SparseCategoricalCrossentropy()])

SparseTopKCategoricalAccuracy class

tensorflow.keras.metrics.SparseTopKCategoricalAccuracy(
    k=5, name="sparse_top_k_categorical_accuracy", dtype=None
)

Computes how often integer targets are in the top K predictions.

Args: k: (Optional) Number of top elements to look at for computing accuracy. Defaults to 5. name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.SparseTopKCategoricalAccuracy(k=1) m.update_state([2, 1], [[0.1, 0.9, 0.8], [0.05, 0.95, 0]]) m.result().numpy() 0.5

m.reset_states() m.update_state([2, 1], [[0.1, 0.9, 0.8], [0.05, 0.95, 0]], ... sample_weight=[0.7, 0.3]) m.result().numpy() 0.3

Usage with compile() API:

model.compile(
  optimizer='sgd',
  loss='mse',
  metrics=[tf.keras.metrics.SparseTopKCategoricalAccuracy()])

SpecificityAtSensitivity class

tensorflow.keras.metrics.SpecificityAtSensitivity(sensitivity, num_thresholds=200, name=None, dtype=None)

Computes best specificity where sensitivity is >= specified value.

Sensitivity measures the proportion of actual positives that are correctly identified as such (tp / (tp + fn)). Specificity measures the proportion of actual negatives that are correctly identified as such (tn / (tn + fp)).

This metric creates four local variables, true_positives, true_negatives, false_positives and false_negatives that are used to compute the specificity at the given sensitivity. The threshold for the given sensitivity value is computed and used to evaluate the corresponding specificity.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

For additional information about specificity and sensitivity, see the following.

Args: sensitivity: A scalar value in range [0, 1]. num_thresholds: (Optional) Defaults to 200. The number of thresholds to use for matching the given sensitivity. name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.SpecificityAtSensitivity(0.5) m.update_state([0, 0, 0, 1, 1], [0, 0.3, 0.8, 0.3, 0.8]) m.result().numpy() 0.66666667

m.reset_states() m.update_state([0, 0, 0, 1, 1], [0, 0.3, 0.8, 0.3, 0.8], ... sample_weight=[1, 1, 2, 2, 2]) m.result().numpy() 0.5

Usage with compile() API:

model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[tf.keras.metrics.SpecificityAtSensitivity()])

SquaredHinge class

tensorflow.keras.metrics.SquaredHinge(name="squared_hinge", dtype=None)

Computes the squared hinge metric between y_true and y_pred.

y_true values are expected to be -1 or 1. If binary (0 or 1) labels are provided we will convert them to -1 or 1.

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.SquaredHinge() m.update_state([[0, 1], [0, 0]], [[0.6, 0.4], [0.4, 0.6]]) m.result().numpy() 1.86

m.reset_states() m.update_state([[0, 1], [0, 0]], [[0.6, 0.4], [0.4, 0.6]], ... sample_weight=[1, 0]) m.result().numpy() 1.46

Usage with compile() API:

model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[tf.keras.metrics.SquaredHinge()])

Sum class

tensorflow.keras.metrics.Sum(name="sum", dtype=None)

Computes the (weighted) sum of the given values.

For example, if values is [1, 3, 5, 7] then the sum is 16. If the weights were specified as [1, 1, 0, 0] then the sum would be 4.

This metric creates one variable, total, that is used to compute the sum of values. This is ultimately returned as sum.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.Sum() m.update_state([1, 3, 5, 7]) m.result().numpy() 16.0

Usage with compile() API:

model.add_metric(tf.keras.metrics.Sum(name='sum_1')(outputs))
model.compile(optimizer='sgd', loss='mse')

TopKCategoricalAccuracy class

tensorflow.keras.metrics.TopKCategoricalAccuracy(k=5, name="top_k_categorical_accuracy", dtype=None)

Computes how often targets are in the top K predictions.

Args: k: (Optional) Number of top elements to look at for computing accuracy. Defaults to 5. name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.TopKCategoricalAccuracy(k=1) m.update_state([[0, 0, 1], [0, 1, 0]], ... [[0.1, 0.9, 0.8], [0.05, 0.95, 0]]) m.result().numpy() 0.5

m.reset_states() m.update_state([[0, 0, 1], [0, 1, 0]], ... [[0.1, 0.9, 0.8], [0.05, 0.95, 0]], ... sample_weight=[0.7, 0.3]) m.result().numpy() 0.3

Usage with compile() API:

model.compile(optimizer='sgd',
              loss='mse',
              metrics=[tf.keras.metrics.TopKCategoricalAccuracy()])

TrueNegatives class

tensorflow.keras.metrics.TrueNegatives(thresholds=None, name=None, dtype=None)

Calculates the number of true negatives.

If sample_weight is given, calculates the sum of the weights of true negatives. This metric creates one local variable, accumulator that is used to keep track of the number of true negatives.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

Args: thresholds: (Optional) Defaults to 0.5. A float value or a python list/tuple of float threshold values in [0, 1]. A threshold is compared with prediction values to determine the truth value of predictions (i.e., above the threshold is true, below is false). One metric value is generated for each threshold value. name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.TrueNegatives() m.update_state([0, 1, 0, 0], [1, 1, 0, 0]) m.result().numpy() 2.0

m.reset_states() m.update_state([0, 1, 0, 0], [1, 1, 0, 0], sample_weight=[0, 0, 1, 0]) m.result().numpy() 1.0

Usage with compile() API:

model.compile(optimizer='sgd',
              loss='mse',
              metrics=[tf.keras.metrics.TrueNegatives()])

TruePositives class

tensorflow.keras.metrics.TruePositives(thresholds=None, name=None, dtype=None)

Calculates the number of true positives.

If sample_weight is given, calculates the sum of the weights of true positives. This metric creates one local variable, true_positives that is used to keep track of the number of true positives.

If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

Args: thresholds: (Optional) Defaults to 0.5. A float value or a python list/tuple of float threshold values in [0, 1]. A threshold is compared with prediction values to determine the truth value of predictions (i.e., above the threshold is true, below is false). One metric value is generated for each threshold value. name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage:

m = tf.keras.metrics.TruePositives() m.update_state([0, 1, 1, 1], [1, 0, 1, 1]) m.result().numpy() 2.0

m.reset_states() m.update_state([0, 1, 1, 1], [1, 0, 1, 1], sample_weight=[0, 0, 1, 0]) m.result().numpy() 1.0

Usage with compile() API:

model.compile(optimizer='sgd',
              loss='mse',
              metrics=[tf.keras.metrics.TruePositives()])

binary_accuracy function

tensorflow.keras.metrics.binary_accuracy(y_true, y_pred, threshold=0.5)

Calculates how often predictions matches binary labels.

Standalone usage:

y_true = [[1], [1], [0], [0]] y_pred = [[1], [1], [0], [0]] m = tf.keras.metrics.binary_accuracy(y_true, y_pred) assert m.shape == (4,) m.numpy() array([1., 1., 1., 1.], dtype=float32)

Args: y_true: Ground truth values. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN]. threshold: (Optional) Float representing the threshold for deciding whether prediction values are 1 or 0.

Returns: Binary accuracy values. shape = [batch_size, d0, .. dN-1]

binary_crossentropy function

tensorflow.keras.metrics.binary_crossentropy(y_true, y_pred, from_logits=False, label_smoothing=0)

Computes the binary crossentropy loss.

Standalone usage:

y_true = [[0, 1], [0, 0]] y_pred = [[0.6, 0.4], [0.4, 0.6]] loss = tf.keras.losses.binary_crossentropy(y_true, y_pred) assert loss.shape == (2,) loss.numpy() array([0.916 , 0.714], dtype=float32)

Args: y_true: Ground truth values. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN]. from_logits: Whether y_pred is expected to be a logits tensor. By default, we assume that y_pred encodes a probability distribution. label_smoothing: Float in [0, 1]. If > 0 then smooth the labels.

Returns: Binary crossentropy loss value. shape = [batch_size, d0, .. dN-1].

categorical_accuracy function

tensorflow.keras.metrics.categorical_accuracy(y_true, y_pred)

Calculates how often predictions matches one-hot labels.

Standalone usage:

y_true = [[0, 0, 1], [0, 1, 0]] y_pred = [[0.1, 0.9, 0.8], [0.05, 0.95, 0]] m = tf.keras.metrics.categorical_accuracy(y_true, y_pred) assert m.shape == (2,) m.numpy() array([0., 1.], dtype=float32)

You can provide logits of classes as y_pred, since argmax of logits and probabilities are same.

Args: y_true: One-hot ground truth values. y_pred: The prediction values.

Returns: Categorical accuracy values.

categorical_crossentropy function

tensorflow.keras.metrics.categorical_crossentropy(y_true, y_pred, from_logits=False, label_smoothing=0)

Computes the categorical crossentropy loss.

Standalone usage:

y_true = [[0, 1, 0], [0, 0, 1]] y_pred = [[0.05, 0.95, 0], [0.1, 0.8, 0.1]] loss = tf.keras.losses.categorical_crossentropy(y_true, y_pred) assert loss.shape == (2,) loss.numpy() array([0.0513, 2.303], dtype=float32)

Args: y_true: Tensor of one-hot true targets. y_pred: Tensor of predicted targets. from_logits: Whether y_pred is expected to be a logits tensor. By default, we assume that y_pred encodes a probability distribution. label_smoothing: Float in [0, 1]. If > 0 then smooth the labels.

Returns: Categorical crossentropy loss value.

deserialize function

tensorflow.keras.metrics.deserialize(config, custom_objects=None)

Deserializes a serialized metric class/function instance.

Arguments: config: Metric configuration. custom_objects: Optional dictionary mapping names (strings) to custom objects (classes and functions) to be considered during deserialization.

Returns: A Keras Metric instance or a metric function.

get function

tensorflow.keras.metrics.get(identifier)

Retrieves a Keras metric as a function/Metric class instance.

The identifier may be the string name of a metric function or class.

metric = tf.keras.metrics.get("categorical_crossentropy") type(metric) metric = tf.keras.metrics.get("CategoricalCrossentropy") type(metric)

You can also specify config of the metric to this function by passing dict containing class_name and config as an identifier. Also note that the class_name must map to a Metric class

identifier = {"class_name": "CategoricalCrossentropy", ... "config": {"from_logits": True}} metric = tf.keras.metrics.get(identifier) type(metric)

Arguments: identifier: A metric identifier. One of None or string name of a metric function/class or metric configuration dictionary or a metric function or a metric class instance

Returns: A Keras metric as a function/ Metric class instance.

Raises: ValueError: If identifier cannot be interpreted.

hinge function

tensorflow.keras.metrics.hinge(y_true, y_pred)

Computes the hinge loss between y_true and y_pred.

loss = mean(maximum(1 - y_true * y_pred, 0), axis=-1)

Standalone usage:

y_true = np.random.choice([-1, 1], size=(2, 3)) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.hinge(y_true, y_pred) assert loss.shape == (2,) assert np.array_equal( ... loss.numpy(), ... np.mean(np.maximum(1. - y_true * y_pred, 0.), axis=-1))

Args: y_true: The ground truth values. y_true values are expected to be -1 or 1. If binary (0 or 1) labels are provided they will be converted to -1 or 1. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN].

Returns: Hinge loss values. shape = [batch_size, d0, .. dN-1].

kl_divergence function

tensorflow.keras.metrics.kl_divergence(y_true, y_pred)

Computes Kullback-Leibler divergence loss between y_true and y_pred.

loss = y_true * log(y_true / y_pred)

See: https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence

Standalone usage:

y_true = np.random.randint(0, 2, size=(2, 3)).astype(np.float64) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.kullback_leibler_divergence(y_true, y_pred) assert loss.shape == (2,) y_true = tf.keras.backend.clip(y_true, 1e-7, 1) y_pred = tf.keras.backend.clip(y_pred, 1e-7, 1) assert np.array_equal( ... loss.numpy(), np.sum(y_true * np.log(y_true / y_pred), axis=-1))

Args: y_true: Tensor of true targets. y_pred: Tensor of predicted targets.

Returns: A Tensor with loss.

Raises: TypeError: If y_true cannot be cast to the y_pred.dtype.

kl_divergence function

tensorflow.keras.metrics.kld(y_true, y_pred)

Computes Kullback-Leibler divergence loss between y_true and y_pred.

loss = y_true * log(y_true / y_pred)

See: https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence

Standalone usage:

y_true = np.random.randint(0, 2, size=(2, 3)).astype(np.float64) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.kullback_leibler_divergence(y_true, y_pred) assert loss.shape == (2,) y_true = tf.keras.backend.clip(y_true, 1e-7, 1) y_pred = tf.keras.backend.clip(y_pred, 1e-7, 1) assert np.array_equal( ... loss.numpy(), np.sum(y_true * np.log(y_true / y_pred), axis=-1))

Args: y_true: Tensor of true targets. y_pred: Tensor of predicted targets.

Returns: A Tensor with loss.

Raises: TypeError: If y_true cannot be cast to the y_pred.dtype.

kl_divergence function

tensorflow.keras.metrics.kullback_leibler_divergence(y_true, y_pred)

Computes Kullback-Leibler divergence loss between y_true and y_pred.

loss = y_true * log(y_true / y_pred)

See: https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence

Standalone usage:

y_true = np.random.randint(0, 2, size=(2, 3)).astype(np.float64) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.kullback_leibler_divergence(y_true, y_pred) assert loss.shape == (2,) y_true = tf.keras.backend.clip(y_true, 1e-7, 1) y_pred = tf.keras.backend.clip(y_pred, 1e-7, 1) assert np.array_equal( ... loss.numpy(), np.sum(y_true * np.log(y_true / y_pred), axis=-1))

Args: y_true: Tensor of true targets. y_pred: Tensor of predicted targets.

Returns: A Tensor with loss.

Raises: TypeError: If y_true cannot be cast to the y_pred.dtype.

log_cosh function

tensorflow.keras.metrics.log_cosh(y_true, y_pred)

Logarithm of the hyperbolic cosine of the prediction error.

log(cosh(x)) is approximately equal to (x ** 2) / 2 for small x and to abs(x) - log(2) for large x. This means that 'logcosh' works mostly like the mean squared error, but will not be so strongly affected by the occasional wildly incorrect prediction.

Standalone usage:

y_true = np.random.random(size=(2, 3)) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.logcosh(y_true, y_pred) assert loss.shape == (2,) x = y_pred - y_true assert np.allclose( ... loss.numpy(), ... np.mean(x + np.log(np.exp(-2. * x) + 1.) - math_ops.log(2.), axis=-1), ... atol=1e-5)

Args: y_true: Ground truth values. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN].

Returns: Logcosh error values. shape = [batch_size, d0, .. dN-1].

log_cosh function

tensorflow.keras.metrics.logcosh(y_true, y_pred)

Logarithm of the hyperbolic cosine of the prediction error.

log(cosh(x)) is approximately equal to (x ** 2) / 2 for small x and to abs(x) - log(2) for large x. This means that 'logcosh' works mostly like the mean squared error, but will not be so strongly affected by the occasional wildly incorrect prediction.

Standalone usage:

y_true = np.random.random(size=(2, 3)) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.logcosh(y_true, y_pred) assert loss.shape == (2,) x = y_pred - y_true assert np.allclose( ... loss.numpy(), ... np.mean(x + np.log(np.exp(-2. * x) + 1.) - math_ops.log(2.), axis=-1), ... atol=1e-5)

Args: y_true: Ground truth values. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN].

Returns: Logcosh error values. shape = [batch_size, d0, .. dN-1].

mean_absolute_error function

tensorflow.keras.metrics.mae(y_true, y_pred)

Computes the mean absolute error between labels and predictions.

loss = mean(abs(y_true - y_pred), axis=-1)

Standalone usage:

y_true = np.random.randint(0, 2, size=(2, 3)) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.mean_absolute_error(y_true, y_pred) assert loss.shape == (2,) assert np.array_equal( ... loss.numpy(), np.mean(np.abs(y_true - y_pred), axis=-1))

Args: y_true: Ground truth values. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN].

Returns: Mean absolute error values. shape = [batch_size, d0, .. dN-1].

mean_absolute_percentage_error function

tensorflow.keras.metrics.mape(y_true, y_pred)

Computes the mean absolute percentage error between y_true and y_pred.

loss = 100 * mean(abs((y_true - y_pred) / y_true), axis=-1)

Standalone usage:

y_true = np.random.random(size=(2, 3)) y_true = np.maximum(y_true, 1e-7) # Prevent division by zero y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.mean_absolute_percentage_error(y_true, y_pred) assert loss.shape == (2,) assert np.array_equal( ... loss.numpy(), ... 100. * np.mean(np.abs((y_true - y_pred) / y_true), axis=-1))

Args: y_true: Ground truth values. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN].

Returns: Mean absolute percentage error values. shape = [batch_size, d0, .. dN-1].

mean_absolute_error function

tensorflow.keras.metrics.mean_absolute_error(y_true, y_pred)

Computes the mean absolute error between labels and predictions.

loss = mean(abs(y_true - y_pred), axis=-1)

Standalone usage:

y_true = np.random.randint(0, 2, size=(2, 3)) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.mean_absolute_error(y_true, y_pred) assert loss.shape == (2,) assert np.array_equal( ... loss.numpy(), np.mean(np.abs(y_true - y_pred), axis=-1))

Args: y_true: Ground truth values. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN].

Returns: Mean absolute error values. shape = [batch_size, d0, .. dN-1].

mean_absolute_percentage_error function

tensorflow.keras.metrics.mean_absolute_percentage_error(y_true, y_pred)

Computes the mean absolute percentage error between y_true and y_pred.

loss = 100 * mean(abs((y_true - y_pred) / y_true), axis=-1)

Standalone usage:

y_true = np.random.random(size=(2, 3)) y_true = np.maximum(y_true, 1e-7) # Prevent division by zero y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.mean_absolute_percentage_error(y_true, y_pred) assert loss.shape == (2,) assert np.array_equal( ... loss.numpy(), ... 100. * np.mean(np.abs((y_true - y_pred) / y_true), axis=-1))

Args: y_true: Ground truth values. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN].

Returns: Mean absolute percentage error values. shape = [batch_size, d0, .. dN-1].

mean_squared_error function

tensorflow.keras.metrics.mean_squared_error(y_true, y_pred)

Computes the mean squared error between labels and predictions.

After computing the squared distance between the inputs, the mean value over the last dimension is returned.

loss = mean(square(y_true - y_pred), axis=-1)

Standalone usage:

y_true = np.random.randint(0, 2, size=(2, 3)) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.mean_squared_error(y_true, y_pred) assert loss.shape == (2,) assert np.array_equal( ... loss.numpy(), np.mean(np.square(y_true - y_pred), axis=-1))

Args: y_true: Ground truth values. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN].

Returns: Mean squared error values. shape = [batch_size, d0, .. dN-1].

mean_squared_logarithmic_error function

tensorflow.keras.metrics.mean_squared_logarithmic_error(y_true, y_pred)

Computes the mean squared logarithmic error between y_true and y_pred.

loss = mean(square(log(y_true + 1) - log(y_pred + 1)), axis=-1)

Standalone usage:

y_true = np.random.randint(0, 2, size=(2, 3)) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.mean_squared_logarithmic_error(y_true, y_pred) assert loss.shape == (2,) y_true = np.maximum(y_true, 1e-7) y_pred = np.maximum(y_pred, 1e-7) assert np.allclose( ... loss.numpy(), ... np.mean( ... np.square(np.log(y_true + 1.) - np.log(y_pred + 1.)), axis=-1))

Args: y_true: Ground truth values. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN].

Returns: Mean squared logarithmic error values. shape = [batch_size, d0, .. dN-1].

mean_squared_error function

tensorflow.keras.metrics.mse(y_true, y_pred)

Computes the mean squared error between labels and predictions.

After computing the squared distance between the inputs, the mean value over the last dimension is returned.

loss = mean(square(y_true - y_pred), axis=-1)

Standalone usage:

y_true = np.random.randint(0, 2, size=(2, 3)) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.mean_squared_error(y_true, y_pred) assert loss.shape == (2,) assert np.array_equal( ... loss.numpy(), np.mean(np.square(y_true - y_pred), axis=-1))

Args: y_true: Ground truth values. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN].

Returns: Mean squared error values. shape = [batch_size, d0, .. dN-1].

mean_squared_logarithmic_error function

tensorflow.keras.metrics.msle(y_true, y_pred)

Computes the mean squared logarithmic error between y_true and y_pred.

loss = mean(square(log(y_true + 1) - log(y_pred + 1)), axis=-1)

Standalone usage:

y_true = np.random.randint(0, 2, size=(2, 3)) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.mean_squared_logarithmic_error(y_true, y_pred) assert loss.shape == (2,) y_true = np.maximum(y_true, 1e-7) y_pred = np.maximum(y_pred, 1e-7) assert np.allclose( ... loss.numpy(), ... np.mean( ... np.square(np.log(y_true + 1.) - np.log(y_pred + 1.)), axis=-1))

Args: y_true: Ground truth values. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN].

Returns: Mean squared logarithmic error values. shape = [batch_size, d0, .. dN-1].

poisson function

tensorflow.keras.metrics.poisson(y_true, y_pred)

Computes the Poisson loss between y_true and y_pred.

The Poisson loss is the mean of the elements of the Tensor y_pred - y_true * log(y_pred).

Standalone usage:

y_true = np.random.randint(0, 2, size=(2, 3)) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.poisson(y_true, y_pred) assert loss.shape == (2,) y_pred = y_pred + 1e-7 assert np.allclose( ... loss.numpy(), np.mean(y_pred - y_true * np.log(y_pred), axis=-1), ... atol=1e-5)

Args: y_true: Ground truth values. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN].

Returns: Poisson loss value. shape = [batch_size, d0, .. dN-1].

Raises: InvalidArgumentError: If y_true and y_pred have incompatible shapes.

serialize function

tensorflow.keras.metrics.serialize(metric)

Serializes metric function or Metric instance.

Arguments: metric: A Keras Metric instance or a metric function.

Returns: Metric configuration dictionary.

sparse_categorical_accuracy function

tensorflow.keras.metrics.sparse_categorical_accuracy(y_true, y_pred)

Calculates how often predictions matches integer labels.

Standalone usage:

y_true = [2, 1] y_pred = [[0.1, 0.9, 0.8], [0.05, 0.95, 0]] m = tf.keras.metrics.sparse_categorical_accuracy(y_true, y_pred) assert m.shape == (2,) m.numpy() array([0., 1.], dtype=float32)

You can provide logits of classes as y_pred, since argmax of logits and probabilities are same.

Args: y_true: Integer ground truth values. y_pred: The prediction values.

Returns: Sparse categorical accuracy values.

sparse_categorical_crossentropy function

tensorflow.keras.metrics.sparse_categorical_crossentropy(y_true, y_pred, from_logits=False, axis=-1)

Computes the sparse categorical crossentropy loss.

Standalone usage:

y_true = [1, 2] y_pred = [[0.05, 0.95, 0], [0.1, 0.8, 0.1]] loss = tf.keras.losses.sparse_categorical_crossentropy(y_true, y_pred) assert loss.shape == (2,) loss.numpy() array([0.0513, 2.303], dtype=float32)

Args: y_true: Ground truth values. y_pred: The predicted values. from_logits: Whether y_pred is expected to be a logits tensor. By default, we assume that y_pred encodes a probability distribution. axis: (Optional) Defaults to -1. The dimension along which the entropy is computed.

Returns: Sparse categorical crossentropy loss value.

sparse_top_k_categorical_accuracy function

tensorflow.keras.metrics.sparse_top_k_categorical_accuracy(y_true, y_pred, k=5)

Computes how often integer targets are in the top K predictions.

Standalone usage:

y_true = [2, 1] y_pred = [[0.1, 0.9, 0.8], [0.05, 0.95, 0]] m = tf.keras.metrics.sparse_top_k_categorical_accuracy( ... y_true, y_pred, k=3) assert m.shape == (2,) m.numpy() array([1., 1.], dtype=float32)

Args: y_true: tensor of true targets. y_pred: tensor of predicted targets. k: (Optional) Number of top elements to look at for computing accuracy. Defaults to 5.

Returns: Sparse top K categorical accuracy value.

squared_hinge function

tensorflow.keras.metrics.squared_hinge(y_true, y_pred)

Computes the squared hinge loss between y_true and y_pred.

loss = mean(square(maximum(1 - y_true * y_pred, 0)), axis=-1)

Standalone usage:

y_true = np.random.choice([-1, 1], size=(2, 3)) y_pred = np.random.random(size=(2, 3)) loss = tf.keras.losses.squared_hinge(y_true, y_pred) assert loss.shape == (2,) assert np.array_equal( ... loss.numpy(), ... np.mean(np.square(np.maximum(1. - y_true * y_pred, 0.)), axis=-1))

Args: y_true: The ground truth values. y_true values are expected to be -1 or 1. If binary (0 or 1) labels are provided we will convert them to -1 or 1. shape = [batch_size, d0, .. dN]. y_pred: The predicted values. shape = [batch_size, d0, .. dN].

Returns: Squared hinge loss values. shape = [batch_size, d0, .. dN-1].

top_k_categorical_accuracy function

tensorflow.keras.metrics.top_k_categorical_accuracy(y_true, y_pred, k=5)

Computes how often targets are in the top K predictions.

Standalone usage:

y_true = [[0, 0, 1], [0, 1, 0]] y_pred = [[0.1, 0.9, 0.8], [0.05, 0.95, 0]] m = tf.keras.metrics.top_k_categorical_accuracy(y_true, y_pred, k=3) assert m.shape == (2,) m.numpy() array([1., 1.], dtype=float32)

Args: y_true: The ground truth values. y_pred: The prediction values. k: (Optional) Number of top elements to look at for computing accuracy. Defaults to 5.

Returns: Top K categorical accuracy value.

Usage in framework

YAML Configuration

Keras metrics

AUC class

threshold values are [0 - 1e-7, 0.5, 1 + 1e-7]

tp = [2, 1, 0], fp = [2, 0, 0], fn = [0, 1, 2], tn = [0, 2, 2]

recall = [1, 0.5, 0], fp_rate = [1, 0, 0]

auc = ((((1+0.5)/2)(1-0))+ (((0.5+0)/2)(0-0))) = 0.75

Accuracy class

BinaryAccuracy class

BinaryCrossentropy class

CategoricalAccuracy class

CategoricalCrossentropy class

EPSILON = 1e-7, y = y_true, y` = y_pred

y` = clip_ops.clip_by_value(output, EPSILON, 1. - EPSILON)

y` = [[0.05, 0.95, EPSILON], [0.1, 0.8, 0.1]]

xent = -sum(y * log(y'), axis = -1)

= -((log 0.95), (log 0.1))

= [0.051, 2.302]

Reduced xent = (0.051 + 2.302) / 2

CategoricalHinge class

CosineSimilarity class

l2_norm(y_true) = [[0., 1.], [1./1.414], 1./1.414]]]

l2_norm(y_pred) = [[1., 0.], [1./1.414], 1./1.414]]]

l2_norm(y_true) . l2_norm(y_pred) = [[0., 0.], [0.5, 0.5]]

result = mean(sum(l2_norm(y_true) . l2_norm(y_pred), axis=1))

= ((0. + 0.) + (0.5 + 0.5)) / 2

FalseNegatives class

FalsePositives class

Hinge class

kl_divergence function

KLDivergence class

LogCoshError class

mean_absolute_error function

mean_absolute_percentage_error function

mean_squared_error function

mean_squared_logarithmic_error function

Mean class

MeanAbsoluteError class

MeanAbsolutePercentageError class

MeanIoU class

cm = [[1, 1],

[1, 1]]

sum_row = [2, 2], sum_col = [2, 2], true_positives = [1, 1]

iou = true_positives / (sum_row + sum_col - true_positives))

result = (1 / (2 + 2 - 1) + 1 / (2 + 2 - 1)) / 2 = 0.33

MeanRelativeError class

metric = mean(|y_pred - y_true| / normalizer)

= mean([1, 1, 4, 5] / [1, 3, 2, 3]) = mean([1, 1/3, 2, 5/3])

= 5/4 = 1.25

MeanSquaredError class

MeanSquaredLogarithmicError class

MeanTensor class

Metric class

Poisson class

Precision class

With top_k=2, it will calculate precision over y_true[:2] and y_pred[:2]

With top_k=4, it will calculate precision over y_true[:4] and y_pred[:4]

PrecisionAtRecall class

Recall class

RecallAtPrecision class

RootMeanSquaredError class

SensitivityAtSpecificity class

SparseCategoricalAccuracy class

SparseCategoricalCrossentropy class

y_true = one_hot(y_true) = [[0, 1, 0], [0, 0, 1]]

logits = log(y_pred)

softmax = exp(logits) / sum(exp(logits), axis=-1)

softmax = [[0.05, 0.95, EPSILON], [0.1, 0.8, 0.1]]

xent = -sum(y * log(softmax), 1)

log(softmax) = [[-2.9957, -0.0513, -16.1181],

[-2.3026, -0.2231, -2.3026]]

y_true * log(softmax) = [[0, -0.0513, 0], [0, 0, -2.3026]]

xent = [0.0513, 2.3026]

Reduced xent = (0.0513 + 2.3026) / 2

SparseTopKCategoricalAccuracy class

SpecificityAtSensitivity class

SquaredHinge class

Sum class

TopKCategoricalAccuracy class