> For the complete documentation index, see [llms.txt](https://www.mlcompendium.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://www.mlcompendium.com/deep-learning/deep-neural-machine-vision.md).

# Deep Neural Machine Vision

Machine vision asks a network to find, name, and outline what is in an image. The page starts with the tools that clean an image collection and with super resolution, then spends most of its length on detection, from the R-CNN family to YOLO, and closes with recognition and segmentation.

The same notes are in [Machine Vision annotation](/data/processing-intro/annotation-and-disagreement.md#machine-vision-annotation) and [Vision](/generative-ai/vision.md).

## TOOLS

Before training anything, the image set itself has to be clean. [Image deduplication](https://github.com/idealo/imagededup) is idealo/imagededup, 😎 Finding duplicate images made easy! The other tool the author lists is Segment anything by Meta, whose demo is kept at the end of the page.

## SUPER RESOLUTION

Once the images are clean, some are simply too small, and super resolution recovers detail from the image itself. [State of the art comparison](http://www.wisdom.weizmann.ac.il/~vision/zssr/) is the project page for "Zero Shot" Super-Resolution using Deep Internal Learning.

## DETECTION

With usable images in hand, detection is the core task: find each object and draw its box. The figure below sets the scene.

The same notes are in [CONVOLUTIONAL NEURAL NET](/deep-learning/convolutional-nets.md#convolutional-neural-net) and [Mix N Match](/generative-ai/mix-n-match.md).

<figure><img src="https://83674056-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Mgd48oS5_duTKOVE_Et%2Fuploads%2Fgit-blob-557f1be9b2f2797b647115002baf0868410203f9%2Fgimg-14a93cb242ce.png?alt=media" alt=""><figcaption><p>Detection</p></figcaption></figure>

The overviews come first. [Review on DL technique applied to semantic segmentation](https://arxiv.org/pdf/1704.06857.pdf) is the review by A. Garcia-Garcia, S. Orts-Escolano, and co-authors. [Mastery on obj detection](https://machinelearningmastery.com/object-recognition-with-deep-learning/) is Jason Brownlee's gentle introduction to object recognition with deep learning, written because beginners find it hard to tell image classification, object localization, and object detection apart when all three get called object recognition.

The frameworks come next. Fair [detectron](https://github.com/facebookresearch/Detectron) is FAIR's research platform for object detection research, implementing popular algorithms like Mask R-CNN and RetinaNet. [Maskrcnn benchmark](https://github.com/facebookresearch/maskrcnn-benchmark) is a fast, modular reference implementation of instance segmentation and object detection algorithms in PyTorch, and its [paper](https://arxiv.org/abs/1703.06870) is Mask R-CNN. [Simpledet - obj detection and instance recognition](https://github.com/TuSimple/simpledet) is a simple and versatile framework for both, and [Mmdetection](https://github.com/open-mmlab/mmdetection) is the OpenMMLab Detection Toolbox and Benchmark.

Detection sits next to separation and segmentation. [Blind image separation](https://www.researchgate.net/publication/3938186_Blind_image_separation_through_kurtosis_maximization) is the paper on blind image separation through kurtosis maximization. [UNET](https://heartbeat.fritz.ai/deep-learning-for-image-segmentation-u-net-architecture-ff17f6e4c1cf) is Fritz's post on U-Net for image segmentation, the process that partitions an image into regions to separate objects and textures. [U^2 Net - using a detection network for pencil drawing generation and segmentation](https://github.com/NathanUA/U-2-Net) is the code for the Pattern Recognition 2020 paper "U^2-Net: Going Deeper with Nested U-Structure for Salient Object Detection." The author's FastAI image segmentation note is kept at the end of the page. The two figures below come from the same detection notes.

<figure><img src="https://83674056-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Mgd48oS5_duTKOVE_Et%2Fuploads%2Fgit-blob-87df391117f5372afcd4ffd4e4a158f8bce6226f%2Fgimg-6e15f36aa9d3.png?alt=media" alt=""><figcaption><p>Detection</p></figcaption></figure>

<figure><img src="https://83674056-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Mgd48oS5_duTKOVE_Et%2Fuploads%2Fgit-blob-2c3f8133116f33bf2feec4a5b93247094a879dba%2Fgimg-8804b92af18e.png?alt=media" alt=""><figcaption><p>Detection</p></figcaption></figure>

The single-shot line of detectors is YOLO, in three papers: [You Only Look Once: Unified, Real-Time Object Detection](https://arxiv.org/abs/1506.02640), then [YOLO9000: Better, Faster, Stronger](https://arxiv.org/abs/1612.08242), then [YOLOv3: An Incremental Improvement](https://arxiv.org/abs/1804.02767). Its code is [YOLO, GitHub](https://github.com/pjreddie/darknet/wiki/YOLO:-Real-Time-Object-Detection), the YOLO: Real Time Object Detection page in pjreddie's darknet.

The two-stage line is the R-CNN family, and its code comes in order. [R-CNN: Regions with Convolutional Neural Network Features, GitHub](https://github.com/rbgirshick/rcnn) is rbgirshick/rcnn, and [Fast R-CNN, GitHub](https://github.com/rbgirshick/fast-rcnn) is rbgirshick/fast-rcnn. For Faster R-CNN, the Python implementation -- see <https://github.com/ShaoqingRen/faster\\_rcnn> for the official MATLAB version - rbgirshick/py-faster-rcnn - is [Faster R-CNN Python Code, GitHub](https://github.com/rbgirshick/py-faster-rcnn).

The papers behind that code, in order, are [Rich feature hierarchies for accurate object detection and semantic segmentation](https://arxiv.org/abs/1311.2524), [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://arxiv.org/abs/1406.4729), [Fast R-CNN](https://arxiv.org/abs/1504.08083), [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://arxiv.org/abs/1506.01497), and [Mask R-CNN](https://arxiv.org/abs/1703.06870).

To read that history as one story, [A Brief History of CNNs in Image Segmentation: From R-CNN to Mask R-CNN](https://blog.athelas.com/a-brief-history-of-cnns-in-image-segmentation-from-r-cnn-to-mask-r-cnn-34ea83205de4), 2017, walks from R-CNN to Mask R-CNN. Lil'Log covers the same ground in [Object Detection for Dummies Part 3: R-CNN Family](https://lilianweng.github.io/lil-log/2017/12/31/object-recognition-for-dummies-part-3.html) and then [Object Detection Part 4: Fast Detection Models](https://lilianweng.github.io/lil-log/2018/12/27/object-detection-part-4.html). For data to practice on, [Ikea ASM](https://ikeaasm.github.io/) is the IKEA Assembly Dataset.

## RECOGNITION

Detection finds objects; recognition names them, and labels can come from where people already put them. [Using image hashtags](https://engineering.fb.com/ml-applications/advancing-state-of-the-art-image-recognition-with-deep-learning-on-hashtags/) is Engineering at Meta on advancing state-of-the-art image recognition with deep learning on hashtags.

## Segmentation

After boxes and names, segmentation outlines each object pixel by pixel, and transformer features can do that too.

The same notes are in [CONVOLUTIONAL NEURAL NET](/deep-learning/convolutional-nets.md#convolutional-neural-net) and [Mix N Match](/generative-ai/mix-n-match.md).

[Vit](https://dino-vit-features.github.io/) is the project page for 'Deep ViT Features as Dense Visual Descriptors.'

## Deprecated links

{% hint style="warning" %}
These links and images no longer work. The original wording is kept here. A same-resource copy, when one was checked, is used above.
{% endhint %}

* Segment anything by Meta. This address no longer opens: <https://segment-anything.com/demo>
* FastAI image segmentation. This address no longer opens: <https://gilberttanner.com/blog/fastai-image-segmentation>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://www.mlcompendium.com/deep-learning/deep-neural-machine-vision.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.