Classification and Object Detection: From Classical Methods to Current Architectures

The previous article established the distinction between classification and detection in terms of output scale: a single verdict for the entire frame versus a label with the object’s location. This article covers the methods used to solve both tasks, from classical algorithms to current architectures, and the factors relevant to implementation.

Problem Definition

Classification takes an image as input and returns a single class label describing the image content. If a photo contains both a dog and a cat, a classifier in the basic problem formulation returns one answer, typically the dominant class.

Object detection is a more detailed task: it locates every object of the target classes in an image and returns, for each one, both a label and the coordinates of the region it occupies, usually as a bounding box. If an image contains three objects, a detector returns three separate results.

These tasks differ in complexity, and the choice between them must be driven by the actual business requirement, not by convention or by the availability of a ready-made solution.

Classical Methods

Before the widespread adoption of deep learning, both tasks were solved by the same general principle: an engineer specified which image features were relevant, and an algorithm computed them and passed them to a classifier. The mechanics of these methods are worth covering in detail, not for historical interest, but because understanding their internal structure determines when they remain the correct choice in practice.

HOG (Histogram of Oriented Gradients) and SVM. HOG constructs a description of an object’s shape as follows: the image is divided into small cells, and within each cell the brightness gradient is computed — the direction and magnitude of local intensity change. A histogram is built within each cell, recording how many gradients point into each of a fixed set of angular bins. Histograms from neighboring cells are grouped into blocks and normalized to reduce the effect of illumination changes. The result is a numerical vector describing shape structure, contours, and silhouette rather than raw pixels. This representation is stable under small illumination changes and minor shifts. The feature vector is passed to a classifier, most commonly a Support Vector Machine (SVM), which constructs a separating hyperplane between object classes in feature space, maximizing the margin between the nearest examples of different classes. The HOG-SVM combination — HOG for shape description, SVM for class separation — was the standard method for tasks such as pedestrian detection for an extended period.

Haar Cascades (Viola-Jones method). Detection requires checking many regions of an image quickly, which called for a different feature set: Haar features, which describe the difference in aggregate brightness between adjacent rectangular regions (for example, a light band above a dark one, characteristic of the eye region on a face). The engineering decision that made the method practical was the integral image: a precomputed data structure that returns the brightness sum for any rectangular image region in constant time, independent of the region’s size. This sharply accelerated Haar feature computation. Selection of the most informative features out of tens of thousands of candidates was performed by AdaBoost, which combines multiple weak classifiers, each only slightly better than random guessing, into a single strong classifier. The final decision was structured as a cascade: the first, simplest and fastest checks rejected the large majority of empty image regions almost instantly, and only a small fraction of candidates reached the more complex, slower checks at later cascade stages. This cascade structure — reject the obviously empty regions quickly, examine only the plausible candidates in detail — made real-time face detection possible on early-2000s processors without any specialized accelerator.

Sliding Window and Multi-Scale Search. Because an object can appear at any location and at any scale within an image, classical detectors applied the classifier repeatedly: a fixed-size window was moved across every position in the image, and the image was then resized (or the window resized) and the pass repeated to find objects at different scales. This produced tens or hundreds of thousands of checks per image, which is why the speed of each individual check was critical — this is the reason for the engineering effort behind the integral image and cascade rejection. When the classifier produced several overlapping detections around the same object, non-maximum suppression was applied, retaining only the single highest-confidence detection per object and discarding the rest as duplicates.

In example below see how sliding window works.

Why Classical Methods Remain a Valid Choice

Classical methods have not become obsolete. The reason is not conservatism but specific engineering constraints regularly encountered in deployment on real hardware rather than in a cloud environment with unbounded resources.

Compute and power budget. A classical algorithm based on HOG or Haar cascades can run on a microcontroller or a simple embedded processor without a GPU or a neural accelerator, consuming single- or double-digit milliwatts. For a battery-powered device expected to operate for months without recharging, this is not a matter of preference but a matter of whether the system can run at all. Neural network inference, even optimized and quantized, in most cases requires substantially more compute and power for the same decision.

Determinism and predictable latency. A classical algorithm executes a fixed, known sequence of arithmetic operations; given identical input and hardware, execution time is predictable and stable across runs. For systems where frame-processing latency is embedded in a real-time control loop, this predictability is often more important than average accuracy — a system that stalls unpredictably in 2% of cases is worse than one that is consistently slightly less accurate but never exceeds its allotted time budget.

Explainability and formal verification. A decision made by a Haar cascade or an HOG-based classifier can be decomposed step by step: which specific feature triggered the result, and at which cascade stage. This matters where the system's behavior must be auditable or certifiable — for example, in industrial safety contexts, or where regulatory requirements prohibit the use of a system whose behavior cannot be fully explained and verified against edge cases.

Independence from training data volume and diversity. A classical algorithm with manually specified features does not require thousands of labeled examples to reach acceptable performance; its parameters can be calibrated for known, fixed installation conditions — a fixed camera angle, known lighting, a limited set of object types. This is particularly valuable in narrow, stable production scenarios where a deep learning approach would require disproportionate engineering effort for data collection and labeling relative to the resulting accuracy gain.

In practice, sound engineering design frequently combines both approaches rather than choosing one exclusively: a classical algorithm serves as a fast, low-cost initial filter that discards clearly empty frames or regions, and a neural network is applied only where actually required, on the remaining, substantially smaller portion of the data. This architecture reduces total computational load while preserving accuracy where it is needed.

Deep Learning Methods

Deep learning changed both tasks by removing the need to hand-design features: the model learns to extract them directly from labeled data. The progression can be divided into three generations.

Convolutional Neural Networks (CNNs) for classification. Rather than an engineer specifying features in advance, a convolutional network learns to extract them automatically, layer by layer — from simple elements (edges, basic textures) in early layers to complex, object-specific features in deeper layers. This substantially improved classification accuracy on large, diverse datasets compared to classical features.

Two-stage detectors. The first generation of deep learning-based detectors (the R-CNN family) solved the task in two steps: an algorithm first proposed a set of candidate regions likely to contain an object, and each candidate region was then classified by a separate network pass. This produced high accuracy but was relatively slow, since each proposed region required a separate forward pass through the network.

Single-stage detectors. The next generation (YOLO-type architectures) combined class prediction and coordinate regression into a single network pass over the entire image, without a separate region-proposal stage. This substantially increased processing speed, at the cost of a modest accuracy reduction in difficult cases, and was the key factor that made real-time object detection practically feasible. This point is addressed in detail in the article on video stream analysis.

Noise and Interference

Before covering evaluation metrics, a separate factor deserves attention, since in practice it affects results as much as model architecture choice: the quality of the image the system actually receives.

Real operating conditions are rarely ideal. An image may contain noise of various origins: weather interference (snow, rain, fog), local distortion from lens or sensor contamination, video compression artifacts, glare from metallic or wet surfaces. Any of these can distort the features a model relies on, causing missed detections or false positives. Classical and deep learning methods are affected differently: classical methods degrade in a more predictable manner, while deep learning methods can behave less predictably if such interference was not represented in the training data.

This can be addressed at several levels; the principle is stated here without further elaboration, since it is a separate, substantial topic in its own right:

• Preprocessing. Noise filtering and image correction before input to the model (covered as the preprocessing stage in the first article).
• Data. Including examples with interference realistic for the specific operating conditions in the training set, rather than only clean images.
• Architecture and training. Techniques that improve model robustness to noise, including deliberate injection of interference during training so the model is not brittle with respect to deviations from ideal conditions.

The practical implication is direct: if a system will operate under conditions where noise and interference are realistic — outdoors, in a facility with dust and vibration, with a camera that will inevitably become dirty — this must be accounted for from the start of the project rather than addressed after deployment, once the system fails to perform as it did in a demonstration.

Evaluation Metrics

Evaluating classification and detection requires different metrics, and confusion between them is a common source of incorrect conclusions about system readiness.

For classification, the baseline metric is accuracy (the proportion of correct answers), which can be misleading with imbalanced classes: if 95% of objects belong to one class, a model that always predicts that class obtains 95% accuracy without solving any actual problem. Precision (the proportion of true positives among all positive predictions) and recall (the proportion of true objects found among all objects present), together with a confusion matrix showing which classes the system conflates, give a more accurate picture.

For detection, the primary metric is IoU (Intersection over Union), which measures the overlap between the predicted region and the ground-truth region. mAP (mean Average Precision) is built on IoU — an averaged accuracy score across classes and confidence thresholds, and the standard metric for comparing detectors.

Common Implementation Errors

Several patterns occur in real projects more often than they should:

Training data that does not reflect real operating conditions. A model trained on carefully captured, well-lit examples can lose accuracy sharply under real conditions — different camera angles, different lighting, objects rarely represented in training data. This is a direct consequence of the point established in the first article of the series: the atypical 20% of cases determines most of a project's cost and complexity.

Task confusion at project outset. The business need is frequently detection (locating and counting specific objects), while the system built is whole-frame classification ("defect present / not present"), because it is simpler and faster to implement. The result exists formally but does not provide the information the business requires.

Ignoring class imbalance. If one class is substantially underrepresented in the labeled data, and this is not accounted for at either the data collection or the training stage, the model will systematically underperform on precisely that rare and often most critical class, such as a rare but significant defect type.

What's Next

This article covered classification and detection: classical features and the sliding window approach, current single-stage detectors, evaluation metrics, and common implementation errors, including a dedicated section on image noise and interference. The next article addresses segmentation, where the output is defined at the level of individual pixels rather than the whole frame or a rectangular region.

If you need help with implementation of Computer Vision systems, contact me via SOFYCOD corporate web site Contact Author, and I can consult you about any questions related to development and implementation of Computer Vision systems.