Key Takeaways
- On MOT17, OC-SORT leads MOTA at 76.0 and IDF1 at 76.4, while ByteTrack reaches 74.1 and DeepSORT trails at 71.4.
- ByteTrack runs over 2000 FPS for tracking logic alone; OC-SORT about 800 FPS; DeepSORT about 200 FPS because of ReID.
- DeepSORT has the fewest ID switches, around 780, but the lowest overall tracking accuracy of the three.
- ByteTrack suits low-compute dense scenes, OC-SORT suits fast nonlinear motion, DeepSORT only suits servers.
ByteTrack vs OC-SORT vs DeepSORT is a choice between three engineering philosophies rather than three versions of the same idea. All three are tracking-by-detection systems built on Kalman prediction plus Hungarian assignment, yet they diverge sharply in matching strategy, motion model and whether appearance features are used. As of 2026 they remain the three reference points for multi-object tracking, and picking the wrong one costs either accuracy or frame rate.
Core mechanism comparison
All three follow the same pipeline: Kalman prediction, cost computation, Hungarian assignment, track lifecycle management. The differences sit in three places: the matching strategy, the motion-model refinement, and whether appearance features are introduced.
| Algorithm | Venue | Core idea | Key innovation | Motion model | Appearance / ReID |
|---|---|---|---|---|---|
| ByteTrack | ECCV 2022 | Associate every detection box, never discard low-score ones | BYTE two-stage strategy: high-score boxes matched first, low-score boxes used to recover occluded or blurred targets | Standard linear Kalman filter | None |
| OC-SORT | CVPR 2023 | Treat real observations as the anchor and correct accumulated filter error | ORU observation re-update, OCM direction momentum and OCR observation recovery address drift under occlusion and nonlinear motion | Observation-centric Kalman filter | None (post-hoc ReID can be added) |
| DeepSORT | ICIP 2017 | Use appearance features to supplement motion matching and preserve ID through long occlusion | Cascade matching, Mahalanobis distance and ReID appearance matching in a three-level strategy | Standard linear Kalman filter | Yes, separate ReID model |
How the matching flows differ
ByteTrack: two-stage matching
- Match high-confidence detection boxes against active tracks by IOU.
- Match low-confidence boxes against the remaining unmatched tracks by IOU again, recovering missed detections.
OC-SORT: four-level layered matching
- Match active tracks using IOU combined with the OCM direction constraint.
- Optionally enable the Byte strategy to fill gaps with low-score boxes.
- Match short-term lost tracks using the last real observation position, which is the OCR mechanism.
- On a successful match, trigger ORU to interpolate and correct the accumulated Kalman error.
DeepSORT: three-level cascade matching
- Cascade by track age, matching first on ReID appearance cosine distance.
- Validate with the Mahalanobis distance as a motion constraint.
- Fall back to IOU matching for targets and detections still unassociated.
Tracking accuracy on MOT17
The figures below come from public benchmarks using the same YOLO detector and the same input resolution, so they are comparable within this table.
| Algorithm | MOTA (higher is better) | HOTA (higher is better) | IDF1 (higher is better) | ID switches (lower is better) |
|---|---|---|---|---|
| ByteTrack | 74.1 | 60.1 | 73.2 | About 1800 |
| OC-SORT | 76.0 | 61.9 | 76.4 | About 1300 |
| DeepSORT | 71.4 | 57.8 | 72.5 | About 780 |
DeepSORT has the fewest ID switches because of its ReID appearance features, yet its overall tracking accuracy is lower than ByteTrack and OC-SORT. The reason is an ageing motion model combined with the extra appearance mismatches that ReID introduces.
Runtime speed
Speed has to be read on two axes: tracking logic alone, without detection, and the end-to-end chain paired with a YOLOv8n detector.
| Algorithm | Tracking only (single CPU) | End to end (mid-range GPU) | End to end (RK3588 edge) | Extra model cost |
|---|---|---|---|---|
| ByteTrack | Over 2000 FPS | 100-150 FPS | 60-80 FPS | None |
| OC-SORT | About 800 FPS | 60-110 FPS | 40-60 FPS | None |
| DeepSORT | About 200 FPS | 30-60 FPS | 15-30 FPS | Loads a ReID model, plus 200 MB memory |
The gap comes from where the compute goes: more than 70 percent of DeepSORT’s cost is ReID feature extraction rather than matching logic, while ByteTrack and OC-SORT are pure geometry and cost very little.
Behaviour by scenario
| Scenario | ByteTrack | OC-SORT | DeepSORT |
|---|---|---|---|
| Dense crowds with many missed detections | Best (low-score recovery) | Very good | Fair |
| Frequent short occlusion (under 1 s) | Fair | Best (ORU plus OCR) | Good |
| Nonlinear motion and fast turns | Poor | Best (OCM direction constraint) | Fair |
| Long full occlusion (over 3 s) | Very poor | Poor | Best (ReID appearance) |
| Small targets with jittery boxes | Fair | Very good | Poor |
| Moving camera and aerial footage | Poor | Fair | Poor |
Engineering fit
Development and ecosystem
| Dimension | ByteTrack | OC-SORT | DeepSORT |
|---|---|---|---|
| Official YOLO support | Native in Ultralytics, one config flag | Native in Ultralytics | No native support, third-party implementations only |
| Code complexity | Minimal, core matching about 200 lines | Moderate, core logic about 500 lines | Complex, feature extraction plus multi-level matching about 1000 lines |
| Language coverage | Python and C++ implementations widely available, largest industrial base | Python and C++ both available, cheap C++ port | Mostly Python, C++ port needs ReID model quantisation |
| Tuning difficulty | Very low, two core thresholds | Moderate, three to four core parameters | High, six or more parameters across motion, appearance and matching |
Edge deployment friendliness
- ByteTrack: the default for edge devices. Pure CPU arithmetic, no extra model, memory under 10 MB, and it runs smoothly on RK3568, Raspberry Pi and similar low-power hardware.
- OC-SORT: the step-up option for edge. Also pure CPU with no extra model and slightly slower than ByteTrack, but occlusion robustness improves by more than 30 percent, which suits RK3588, Jetson Orin NX and above.
- DeepSORT: expensive to deploy at the edge. A separate ReID model is required, ordinary NPUs and MCUs struggle with feature extraction, and quantisation costs noticeable accuracy, so it is generally not recommended for pure edge use.
Extensibility
- ByteTrack: weak. The framework is fixed and adding ReID or similar extensions natively is difficult.
- OC-SORT: strong. Post-hoc ReID and camera motion compensation can be added cleanly, and Ultralytics ships a Deep OC-SORT variant that pairs OC-SORT with a lightweight ReID model.
- DeepSORT: moderate. ReID is built in, but the motion model is old and there is little room to optimise the foundation.
Strengths and weaknesses
ByteTrack
- Strengths: extremely fast with very low compute, the most mature engineering base, low-score recovery that clearly beats first-generation SORT in dense scenes, zero extra dependencies, and near-zero development and tuning cost through native Ultralytics support.
- Weaknesses: pure IOU matching so IDs switch frequently after occlusion, high loss rate under nonlinear motion or fast turns, and no appearance feature so long occlusion cannot be bridged at all.
OC-SORT
- Strengths: the performance ceiling for pure motion tracking, far ahead of ByteTrack under occlusion and nonlinear motion, no extra model and only slightly more compute, flexible ReID extension, and observation-centric design that is more robust to detection jitter and small targets.
- Weaknesses: slightly lower recall than ByteTrack in extremely dense scenes when the Byte strategy is off, fewer industrial deployments because it is newer, and still a pure motion model so very long occlusion beyond 5 seconds remains unsolved.
DeepSORT
- Strengths: native ReID with the best ID continuity under long occlusion, a mature cascade strategy that is stable when appearance differs clearly, and high industrial recognition with a large installed base.
- Weaknesses: slow and compute-heavy, hard to deploy at the edge, lower overall MOTA than ByteTrack and OC-SORT, sensitive to lighting and viewpoint change, an ageing motion model, and high complexity for tuning and maintenance.
Selection guide
Decision path
- Do you need to survive occlusion longer than one second?
- No: is compute constrained?
- Yes (RK3568, Raspberry Pi, low-power CPU): ByteTrack.
- No (RK3588, Jetson, mid-range GPU): is the motion violent?
- Yes (drones, sports, fast turns): OC-SORT.
- No (standard security, people counting): ByteTrack, best value.
- Yes: how long is the occlusion?
- 1-3 seconds: OC-SORT plus post-hoc lightweight ReID, the recommended path.
- Over 3 seconds: is compute plentiful?
- Yes (server, high-end GPU): DeepSORT, or OC-SORT with cascade ReID.
- No (edge): OC-SORT with ReID reusing the YOLO backbone.
- No: is compute constrained?
Recommendation by application
| Application | First choice | Alternative | Why |
|---|---|---|---|
| Edge people or vehicle counting | ByteTrack | OC-SORT | Low compute, good dense-scene recall, ID switches barely matter for counting |
| Security monitoring, corridor and elevator occlusion | OC-SORT | ByteTrack with lightweight ReID | Frequent short occlusion that pure motion already handles, no extra compute |
| Drone and counter-UAV tracking | OC-SORT | – | Fast nonlinear targets where the OCM direction constraint clearly helps |
| Person trajectory tracing and behaviour analysis | OC-SORT with post-hoc ReID | DeepSORT | High ID continuity requirement, layered matching balances speed and accuracy |
| Server-side offline video analysis | DeepSORT | BoT-SORT | Compute is plentiful, so maximum ID stability wins |
| Industrial line and product counting | SORT or ByteTrack | – | No occlusion and regular motion, the simplest option is enough |
Four common misconceptions
- ReID does not automatically mean higher accuracy. It improves ID continuity, not overall tracking accuracy. When detection quality is good, a pure motion tracker matches more accurately.
- Newer does not mean replace it. ByteTrack is older but still unbeatable on value in dense and low-compute scenarios, and it is enough for most industrial projects.
- Do not deploy DeepSORT blindly at the edge. Its cost is dominated by ReID feature extraction, so the return on edge hardware is poor. Prefer OC-SORT with lightweight ReID.
- Detection sets the ceiling. Missed detections, false positives and box jitter from the YOLO model affect tracking far more than the choice of tracker. Improving detection is the first priority.
Have questions about this article? Feel free to contact us at [email protected] — we’re happy to help!
Frequently Asked Questions
Which tracker has the best accuracy on MOT17?
OC-SORT, with MOTA 76.0, HOTA 61.9 and IDF1 76.4. ByteTrack follows at 74.1 MOTA, and DeepSORT trails at 71.4 despite having the fewest ID switches.
Is ByteTrack or OC-SORT faster?
ByteTrack, by a wide margin: over 2000 FPS for tracking logic alone against about 800 FPS for OC-SORT. End to end with YOLOv8n on an RK3588, that is 60-80 FPS versus 40-60 FPS.
Why does DeepSORT have fewer ID switches but lower accuracy?
Its ReID appearance features preserve identity through occlusion, which cuts switches to about 780. But the underlying motion model is old and appearance matching introduces its own false matches, so overall accuracy ends up lower.
Can ByteTrack handle long occlusion?
No. ByteTrack has no appearance feature, so once a target is fully occluded for more than a few frames the ID cannot be recovered. Use OC-SORT with ReID, or DeepSORT, when long occlusion is expected.
What should I run on an RK3588 for drone tracking?
OC-SORT. It is pure CPU with no extra model, holds 40-60 FPS end to end with YOLOv8n, and its OCM direction constraint handles the fast nonlinear motion typical of aerial targets.
About Aomway
Aomway is a radio-frequency engineering company that designs and manufactures long-range FPV video transmitters, antennas, goggles and UAV communication links, and it publishes technical notes on perception, tracking and embedded control. Its hardware is built for builders who need dependable links at range, and its documentation aims to explain the engineering behind each product.
