deep_sort package¶

deep_sort.detection module¶

class deep_sort.detection.Detection(tlwh, confidence, class_name, feature)¶

Bases: object

This class represents a bounding box detection in a single image.

Parameters

tlwh (array_like) – Bounding box in format (x, y, w, h).
confidence (float) – Detector confidence score.
feature (array_like) – A feature vector that describes the object contained in this image.

tlwh¶

Bounding box in format (top left x, top left y, width, height).

Type: ndarray

confidence¶

Detector confidence score.

Type: ndarray

class_name¶

Detector class.

Type: ndarray

feature¶

A feature vector that describes the object contained in this image.

Type: ndarray | NoneType

get_class()¶

to_tlbr()¶: Convert bounding box to format (min x, min y, max x, max y), i.e., (top left, bottom right).

to_xyah()¶: Convert bounding box to format (center x, center y, aspect ratio, height), where the aspect ratio is width / height.

deep_sort.iou_matching module¶

deep_sort.iou_matching.iou(bbox, candidates)¶

Computes intersection over union.

Parameters

bbox (ndarray) – A bounding box in format (top left x, top left y, width, height).
candidates (ndarray) – A matrix of candidate bounding boxes (one per row) in the same format as bbox.

Returns

The intersection over union in [0, 1] between the bbox and each candidate. A higher score means a larger fraction of the bbox is occluded by the candidate.

Return type

ndarray

deep_sort.iou_matching.iou_cost(tracks, detections, track_indices=None, detection_indices=None)¶

An intersection over union distance metric.

Parameters

tracks (List[deep_sort.track.Track]) – A list of tracks.
detections (List[deep_sort.detection.Detection]) – A list of detections.
track_indices (Optional[List[int]]) – A list of indices to tracks that should be matched. Defaults to all tracks.
detection_indices (Optional[List[int]]) – A list of indices to detections that should be matched. Defaults to all detections.

Returns

Returns a cost matrix of shape len(track_indices), len(detection_indices) where entry (i, j) is 1 - iou(tracks[track_indices[i]], detections[detection_indices[j]]).

Return type

ndarray

deep_sort.kalman_filter module¶

class deep_sort.kalman_filter.KalmanFilter¶

Bases: object

A simple Kalman filter for tracking bounding boxes in image space.

The 8-dimensional state space

x, y, a, h, vx, vy, va, vh

contains the bounding box center position (x, y), aspect ratio a, height h, and their respective velocities.

Object motion follows a constant velocity model. The bounding box location (x, y, a, h) is taken as direct observation of the state space (linear observation model).

gating_distance(mean, covariance, measurements, only_position=False)¶

Compute gating distance between state distribution and measurements.

A suitable distance threshold can be obtained from chi2inv95. If only_position is False, the chi-square distribution has 4 degrees of freedom, otherwise 2.

Parameters

mean (ndarray) – Mean vector over the state distribution (8 dimensional).
covariance (ndarray) – Covariance of the state distribution (8x8 dimensional).
measurements (ndarray) – An Nx4 dimensional matrix of N measurements, each in format (x, y, a, h) where (x, y) is the bounding box center position, a the aspect ratio, and h the height.
only_position (Optional[bool]) – If True, distance computation is done with respect to the bounding box center position only.

Returns

Returns an array of length N, where the i-th element contains the squared Mahalanobis distance between (mean, covariance) and measurements[i].

Return type

ndarray

initiate(measurement)¶

Create track from unassociated measurement.

Parameters: measurement (ndarray) – Bounding box coordinates (x, y, a, h) with center position (x, y), aspect ratio a, and height h.
Returns: Returns the mean vector (8 dimensional) and covariance matrix (8x8 dimensional) of the new track. Unobserved velocities are initialized to 0 mean.
Return type: (ndarray, ndarray)

predict(mean, covariance)¶

Run Kalman filter prediction step.

Parameters

mean (ndarray) – The 8 dimensional mean vector of the object state at the previous time step.
covariance (ndarray) – The 8x8 dimensional covariance matrix of the object state at the previous time step.

Returns

Returns the mean vector and covariance matrix of the predicted state. Unobserved velocities are initialized to 0 mean.

Return type

(ndarray, ndarray)

project(mean, covariance)¶

Project state distribution to measurement space.

Parameters

mean (ndarray) – The state’s mean vector (8 dimensional array).
covariance (ndarray) – The state’s covariance matrix (8x8 dimensional).

Returns

Returns the projected mean and covariance matrix of the given state estimate.

Return type

(ndarray, ndarray)

update(mean, covariance, measurement)¶

Run Kalman filter correction step.

Parameters

mean (ndarray) – The predicted state’s mean vector (8 dimensional).
covariance (ndarray) – The state’s covariance matrix (8x8 dimensional).
measurement (ndarray) – The 4 dimensional measurement vector (x, y, a, h), where (x, y) is the center position, a the aspect ratio, and h the height of the bounding box.

Returns

Returns the measurement-corrected state distribution.

Return type

(ndarray, ndarray)

deep_sort.linear_assignment module¶

deep_sort.linear_assignment.gate_cost_matrix(kf, cost_matrix, tracks, detections, track_indices, detection_indices, gated_cost=100000.0, only_position=False)¶

Invalidate infeasible entries in cost matrix based on the state distributions obtained by Kalman filtering.

Parameters

kf (The Kalman filter.) –
cost_matrix (ndarray) – The NxM dimensional cost matrix, where N is the number of track indices and M is the number of detection indices, such that entry (i, j) is the association cost between tracks[track_indices[i]] and detections[detection_indices[j]].
tracks (List[track.Track]) – A list of predicted tracks at the current time step.
detections (List[detection.Detection]) – A list of detections at the current time step.
track_indices (List[int]) – List of track indices that maps rows in cost_matrix to tracks in tracks (see description above).
detection_indices (List[int]) – List of detection indices that maps columns in cost_matrix to detections in detections (see description above).
gated_cost (Optional[float]) – Entries in the cost matrix corresponding to infeasible associations are set this value. Defaults to a very large value.
only_position (Optional[bool]) – If True, only the x, y position of the state distribution is considered during gating. Defaults to False.

Returns

Returns the modified cost matrix.

Return type

ndarray

deep_sort.linear_assignment.matching_cascade(distance_metric, max_distance, cascade_depth, tracks, detections, track_indices=None, detection_indices=None)¶

Run matching cascade.

Parameters

distance_metric (Callable[List[Track], List[Detection], List[int], List[int]) -> ndarray) – The distance metric is given a list of tracks and detections as well as a list of N track indices and M detection indices. The metric should return the NxM dimensional cost matrix, where element (i, j) is the association cost between the i-th track in the given track indices and the j-th detection in the given detection indices.
max_distance (float) – Gating threshold. Associations with cost larger than this value are disregarded.
cascade_depth (int) – The cascade depth, should be se to the maximum track age.
tracks (List[track.Track]) – A list of predicted tracks at the current time step.
detections (List[detection.Detection]) – A list of detections at the current time step.
track_indices (Optional[List[int]]) – List of track indices that maps rows in cost_matrix to tracks in tracks (see description above). Defaults to all tracks.
detection_indices (Optional[List[int]]) – List of detection indices that maps columns in cost_matrix to detections in detections (see description above). Defaults to all detections.

Returns

Returns a tuple with the following three entries: * A list of matched track and detection indices. * A list of unmatched track indices. * A list of unmatched detection indices.

Return type

(List[(int, int)], List[int], List[int])

deep_sort.linear_assignment.min_cost_matching(distance_metric, max_distance, tracks, detections, track_indices=None, detection_indices=None)¶

Solve linear assignment problem.

Parameters

distance_metric (Callable[List[Track], List[Detection], List[int], List[int]) -> ndarray) – The distance metric is given a list of tracks and detections as well as a list of N track indices and M detection indices. The metric should return the NxM dimensional cost matrix, where element (i, j) is the association cost between the i-th track in the given track indices and the j-th detection in the given detection_indices.
max_distance (float) – Gating threshold. Associations with cost larger than this value are disregarded.
tracks (List[track.Track]) – A list of predicted tracks at the current time step.
detections (List[detection.Detection]) – A list of detections at the current time step.
track_indices (List[int]) – List of track indices that maps rows in cost_matrix to tracks in tracks (see description above).
detection_indices (List[int]) – List of detection indices that maps columns in cost_matrix to detections in detections (see description above).

Returns

Returns a tuple with the following three entries: * A list of matched track and detection indices. * A list of unmatched track indices. * A list of unmatched detection indices.

Return type

(List[(int, int)], List[int], List[int])

deep_sort.nn_matching module¶

class deep_sort.nn_matching.NearestNeighborDistanceMetric(metric, matching_threshold, budget=None)¶

Bases: object

A nearest neighbor distance metric that, for each target, returns the closest distance to any sample that has been observed so far.

Parameters

metric (str) – Either “euclidean” or “cosine”.
matching_threshold (float) – The matching threshold. Samples with larger distance are considered an invalid match.
budget (Optional[int]) – If not None, fix samples per class to at most this number. Removes the oldest samples when the budget is reached.

samples¶

A dictionary that maps from target identities to the list of samples that have been observed so far.

Type: Dict[int -> List[ndarray]]

distance(features, targets)¶

Compute distance between features and targets.

Parameters

features (ndarray) – An NxM matrix of N features of dimensionality M.
targets (List[int]) – A list of targets to match the given features against.

Returns

Returns a cost matrix of shape len(targets), len(features), where element (i, j) contains the closest squared distance between targets[i] and features[j].

Return type

ndarray

partial_fit(features, targets, active_targets)¶

Update the distance metric with new data.

Parameters

features (ndarray) – An NxM matrix of N features of dimensionality M.
targets (ndarray) – An integer array of associated target identities.
active_targets (List[int]) – A list of targets that are currently present in the scene.

deep_sort.preprocessing module¶

deep_sort.preprocessing.non_max_suppression(boxes, classes, max_bbox_overlap, scores=None)¶

Suppress overlapping detections.

Original code from [1] has been adapted to include confidence score.

[1] http://www.pyimagesearch.com/2015/02/16/: faster-non-maximum-suppression-python/

Parameters

boxes (ndarray) – Array of ROIs (x, y, width, height).
max_bbox_overlap (float) – ROIs that overlap more than this values are suppressed.
scores (Optional[array_like]) – Detector confidence score.

Returns

Returns indices of detections that have survived non-maxima suppression.

Return type

List[int]

deep_sort.track module¶

class deep_sort.track.Track(mean, covariance, track_id, n_init, max_age, feature=None, class_name=None)¶

Bases: object

A single target track with state space (x, y, a, h) and associated velocities, where (x, y) is the center of the bounding box, a is the aspect ratio and h is the height.

Parameters

mean (ndarray) – Mean vector of the initial state distribution.
covariance (ndarray) – Covariance matrix of the initial state distribution.
track_id (int) – A unique track identifier.
n_init (int) – Number of consecutive detections before the track is confirmed. The track state is set to Deleted if a miss occurs within the first n_init frames.
max_age (int) – The maximum number of consecutive misses before the track state is set to Deleted.
feature (Optional[ndarray]) – Feature vector of the detection this track originates from. If not None, this feature is added to the features cache.

mean¶

Mean vector of the initial state distribution.

Type: ndarray

covariance¶

Covariance matrix of the initial state distribution.

Type: ndarray

track_id¶

A unique track identifier.

Type: int

hits¶

Total number of measurement updates.

Type: int

age¶

Total number of frames since first occurance.

Type: int

time_since_update¶

Total number of frames since last measurement update.

Type: int

state¶

The current track state.

Type: TrackState

features¶

A cache of features. On each measurement update, the associated feature vector is added to this list.

Type: List[ndarray]

get_class()¶

is_confirmed()¶: Returns True if this track is confirmed.

is_deleted()¶: Returns True if this track is dead and should be deleted.

is_tentative()¶: Returns True if this track is tentative (unconfirmed).

mark_missed()¶: Mark this track as missed (no association at the current time step).

predict(kf)¶

Propagate the state distribution to the current time step using a Kalman filter prediction step.

Parameters: kf (kalman_filter.KalmanFilter) – The Kalman filter.

to_tlbr()¶

Get current position in bounding box format (min x, miny, max x, max y).

Returns: The bounding box.
Return type: ndarray

to_tlwh()¶

Get current position in bounding box format (top left x, top left y, width, height).

Returns: The bounding box.
Return type: ndarray

update(kf, detection)¶

Perform Kalman filter measurement update step and update the feature cache.

Parameters

kf (kalman_filter.KalmanFilter) – The Kalman filter.
detection (Detection) – The associated detection.

class deep_sort.track.TrackState¶

Bases: object

Enumeration type for the single target track state. Newly created tracks are classified as tentative until enough evidence has been collected. Then, the track state is changed to confirmed. Tracks that are no longer alive are classified as deleted to mark them for removal from the set of active tracks.

Confirmed = 2¶

Deleted = 3¶

Tentative = 1¶

deep_sort.tracker module¶

class deep_sort.tracker.Tracker(metric, max_iou_distance=0.7, max_age=60, n_init=3)¶

Bases: object

This is the multi-target tracker.

Parameters

metric (nn_matching.NearestNeighborDistanceMetric) – A distance metric for measurement-to-track association.
max_age (int) – Maximum number of missed misses before a track is deleted.
n_init (int) – Number of consecutive detections before the track is confirmed. The track state is set to Deleted if a miss occurs within the first n_init frames.

metric¶

The distance metric used for measurement to track association.

Type: nn_matching.NearestNeighborDistanceMetric

max_age¶

Maximum number of missed misses before a track is deleted.

Type: int

n_init¶

Number of frames that a track remains in initialization phase.

Type: int

kf¶

A Kalman filter to filter target trajectories in image space.

Type: kalman_filter.KalmanFilter

tracks¶

The list of active tracks at the current time step.

Type: List[Track]

predict()¶

Propagate track state distributions one time step forward.

This function should be called once every time step, before update.

update(detections)¶

Perform measurement update and track management.

Parameters: detections (List[deep_sort.detection.Detection]) – A list of detections at the current time step.