Thermal Image Processing Explained: Why Two 640×512 Thermal Sensors Can Produce Completely Different Images

Thermal Academy · Core Curriculum

Picture two hunters standing shoulder to shoulder on the same paddock, glassing the same tree line on the same still night. Both are running a thermal scope with a 640×512 sensor. Both spec sheets quote a sensitivity (NETD) under 20 mK. On paper, the two optics are twins.

Yet when they swap and look through each other’s scope, the difference is obvious in a heartbeat. One image is crisp, clean and easy to read — the fox at the fence line is unmistakably a fox. The other is softer, noisier, flatter; the same animal is a warm smudge you have to work to identify. Same resolution. Same sensitivity. Wildly different picture.

How is that possible? Because a thermal sensor doesn’t hand you a photograph. It hands you a stream of raw temperature measurements — and something has to turn those numbers into an image a human can actually use. That “something” is image processing, and it is the most under-appreciated part of any thermal optic. Two makers can buy the identical sensor and build wildly different scopes around it, because the sensor is only the first link in a long chain.

This guide walks that entire chain, from infrared energy to the picture in your eye, and explains — in plain language, with no engineering degree required — exactly what the processing engine does and why it, more than any single spec, decides what you see. By the end, one idea should stick above all others.

What Is Thermal Image Processing?

A thermal sensor — an uncooled microbolometer — does one job: it measures how much infrared energy lands on each of its pixels and reports that as a number. A 640×512 sensor produces a grid of roughly 328,000 of these numbers, dozens of times a second. That’s it. It is a spreadsheet of temperatures, not a picture.

Left completely raw, that data is close to useless to a human eye. The differences we care about — a 34°C animal against 31°C grass — are a tiny fraction of the total range the sensor can measure, so a raw frame looks flat, grey and lifeless. Thermal image processing is the software that turns that raw temperature grid into a clear, high-contrast, readable image, in real time, frame after frame.

If you’ve ever shot a camera in RAW, you already understand this. A RAW file straight off a camera sensor looks dull and flat — it isn’t meant to be the final photo; it’s the raw ingredients. The polished image comes from processing: white balance, contrast curves, sharpening, noise reduction. Thermal is exactly the same — except the “editing” happens automatically, instantly, inside the scope, every single frame.

The Thermal Imaging Pipeline

Before we get into what the processing engine does, it’s worth seeing the whole journey a thermal image takes — because image quality is an emergent property of the entire chain, not the output of any one component. Heat leaves your target, and a series of stages each shape the final result before it reaches your eye. A weak link anywhere limits everything downstream.

The full chain, from infrared energy to your eye. The sensor sits early in the line; the processing and enhancement stages — highlighted — are where a good maker turns raw data into a clean, usable picture.

1. Infrared radiation enters the lens

Everything warmer than absolute zero radiates infrared energy. Your target — animal, person, engine block — is constantly emitting long-wave infrared that the naked eye can’t see. That energy travels to the optic and meets the front lens.

2. Germanium optics focus the energy

Thermal lenses aren’t made of glass — ordinary glass blocks infrared. They’re made of germanium, a metallic element transparent to long-wave IR. The germanium objective gathers and focuses that incoming energy onto the sensor, exactly as a camera lens focuses light onto film. Lens quality here already shapes the ceiling of the final image: a soft or poorly-coated lens delivers less contrast to the sensor before processing even begins.

3. The sensor measures temperature differences

Each pixel on the microbolometer heats up very slightly according to the IR energy focused onto it, and the electronics read that as a temperature value. The sensor isn’t taking a photo; it’s building a live grid of temperatures — the raw material for everything that follows.

4. Raw data is digitised

Those analogue readings are converted into digital numbers — a frame of raw thermal data. At this moment the “image” exists only as a flat, noisy array of values. If you displayed it directly, you’d struggle to make anything out. It needs work.

5. Image processing begins

This is the heart of the optic. A dedicated processor runs the raw frame through a stack of algorithms — balancing the scene, stretching contrast, suppressing noise, sharpening edges, correcting the sensor’s own quirks — and does it fast enough to keep up with a live 50 Hz video feed. This single stage is where two identical sensors part ways, and it’s what the rest of this article is about.

6. The display renders the final image

Finally the processed frame is drawn on the scope’s display — usually an OLED — and only now does light reach your eye. The display’s resolution, brightness and colour accuracy are the last gate: a superb processed image still has to survive a mediocre screen. The picture you judge the whole optic by is the sum of every stage before it.

Why Raw Thermal Images Look Terrible

It’s worth being blunt about just how poor unprocessed thermal data is, because it explains why the processing engine matters so much. If you could tap the sensor’s output directly, before any software touched it, you’d see something close to unusable. There are four reasons.

Flat contrast. The sensor measures a wide span of temperatures, but the things you care about occupy a sliver of it. A warm animal might be only two or three degrees above the ground it’s standing on. Spread across the sensor’s full range, that difference is almost invisible — everything reads as a similar shade of grey.

Image noise. Every microbolometer has electronic noise. In a raw frame that shows up as a constant, shimmering speckle — random pixels flickering brighter and darker than they should — which smothers faint targets.

Low dynamic range on screen. The scene may contain both a very hot object and a very cold one at once. Mapped naively, the hot object blows out to pure white and everything else crushes to black, hiding most of the picture.

Poor edge definition. Raw thermal data is inherently soft. Without processing, the boundary between an animal and its background is blurry and indistinct, so shapes are hard to identify even when you can see that something is there.

The identical sensor frame, raw versus processed. Raw thermal data isn’t designed for human eyes — it’s designed to be processed. Everything that makes the right-hand image readable was added by software. (Illustrative.)

What Does the Processing Engine Actually Do?

“Image processing” sounds like a single step. It’s really a stack of distinct algorithms, each fixing a specific problem, run in sequence on every frame. Here are the main ones, in roughly the order they matter.

A simplified processing pipeline. The exact order and techniques vary by maker — and that variation is a big part of why two scopes with the same sensor look different.

Automatic Gain Control (AGC)

The sensor can measure a far wider temperature range than any screen can show at once. AGC is the algorithm that decides which slice of that range to map to the visible image, and constantly rebalances it as the scene changes — swing from cold sky to a warm engine block and AGC re-levels so the picture stays usable. Good AGC is invisible; poor AGC leaves the image washed out or too dark as conditions shift.

Histogram Equalisation

Before contrast can be enhanced, the data usually has to be re-spread. Picture every temperature value in a frame plotted on a graph — a histogram. In a typical thermal scene they bunch into a narrow spike, because most of the picture sits at a similar temperature — and that bunching is precisely why the raw image looks flat and grey. Histogram equalisation takes that crowded spike and spreads it across the full available range of brightness, so tiny temperature differences that were sitting on top of each other pull apart and become visible. It’s the groundwork the rest of the contrast work is built on — and doing it well, locally and scene-by-scene without dragging up noise, is a real craft.

Contrast Enhancement

This is the single biggest driver of a thermal image’s “pop.” Because the temperatures we care about are so close together, the software deliberately stretches and separates them — pushing a 34°C animal apart from 32°C grass so your eye can tell them apart. Techniques like histogram equalisation and local-area contrast do this cleverly, brightening the target without blowing out the rest of the scene.

Edge Enhancement

Also called digital detail enhancement, this sharpens the perceived boundaries between temperature regions so shapes read crisply. Tuned well, an animal snaps into focus against its background. Overdone, it produces ugly halos and amplifies noise — so this is very much a craft, not a slider turned to maximum.

Noise Reduction

Spatial noise reduction smooths the random pixel-to-pixel speckle within a single frame. The trick is restraint: too little and the image shimmers; too much and fine detail smears away with the noise. Balancing suppression against detail retention is one of the hardest tuning problems in the whole engine.

Temporal Filtering

A second, powerful noise weapon: averaging information across successive frames. Because noise is random and the scene is (mostly) steady, blending frames cancels a lot of speckle and dramatically cleans a static image. The catch is motion — average too aggressively and a running animal smears or ghosts. Makers tune how much temporal filtering to apply, and often ease it off when they detect movement.

Dead Pixel Correction

No sensor is perfect; a handful of pixels are always defective and would otherwise appear as fixed bright or dark dots. The software maps these at the factory and replaces each with a value interpolated from its neighbours, so you never see them. A well-maintained map keeps the image clean over the optic’s life.

Non-Uniformity Correction (NUC)

Every pixel responds slightly differently, and the sensor’s own heat drifts as the electronics warm up — left uncorrected, this shows as a faint fixed pattern or a slowly degrading image. NUC re-levels the whole array so it stays even, frequently using a quick internal shutter (the brief “click” and freeze you notice) to grab a fresh reference. It’s important enough to warrant its own full guide, coming to Thermal Academy.

Dynamic Range Optimisation

Real scenes are brutal: a hot exhaust and cold night sky in the same frame. Naive mapping loses one end or the other. Dynamic-range optimisation (a form of local tone-mapping) keeps detail visible across that huge span at once, so you can read the warm animal and still see texture in the cool background instead of a white blob on black.

Sitting on top of all of this is your choice of colour palette — white-hot, black-hot, red-hot and the rest. Palettes are a processing decision too: they map temperatures to colours or shades, and the right one can make a target jump out in conditions where another hides it. Makers add and refine palettes over time, and a good one is a genuine aid, not just decoration. (A full palette guide is coming to Thermal Academy.)

Why Two 640×512 Sensors Can Look Completely Different

Now we can answer the question we opened with. Take two thermal scopes. Both use the same 640×512 sensor, the same 12µm pixel pitch, the same sub-20mK NETD. On the spec sheet they are identical. Put them side by side in the field and one is clearly better. Here’s where that gap comes from.

Two optics, one sensor design, two engines. The hardware sets the ceiling; the software decides how close you get to it.

Four things differ between our two scopes, none of them on the spec sheet:

The algorithms themselves. One maker may have far better contrast, noise-reduction and edge-enhancement routines than the other. This is intellectual property built up over years, and it’s the biggest single differentiator.

Processing power. Sophisticated algorithms need a capable processor to run in real time at full frame rate. A weaker chip forces compromises — simpler routines, more lag, or noise reduction that has to be dialled back to keep up.

Software tuning. Even identical algorithms can be tuned well or badly. Getting the balance between sharpness and noise, or between temporal smoothing and motion clarity, takes enormous amounts of field testing. Good tuning is the difference between “technically correct” and “looks great.”

Display calibration. The last stage. Two scopes can produce an identical processed frame and still look different because their displays differ in resolution, brightness, contrast and colour accuracy. A beautifully processed image still has to survive the screen.

Put it concretely. Both makers buy the same sensor. Maker A pairs it with a powerful processor and a decade of tuning: contrast that lifts a fox cleanly off cool grass, temporal noise reduction that eases off the instant the animal moves, a crisp, well-calibrated OLED. Maker B drops the same sensor onto a cheaper processor running stock, off-the-shelf algorithms at default settings. Both boxes read “640×512, under 20 mK.” In the field, A shows you a fox; B shows you a warm blur that might be a fox. Same sensor — different company behind the software.

Image Processing vs Hardware

It helps to be clear about who does what in a thermal optic. Each part has one job, and the finished image is the relay of all four working together — no single one “is” the image quality.

Lens, sensor, processor, display — a relay. Spec sheets describe the lens and sensor in detail and the processor barely at all, which is exactly why the processor’s contribution is so easy to overlook.

Inside the Engine: The Processor and the Latency Trade-off

We’ve been saying “the processing engine” as if it were one thing. It’s worth a moment on what actually runs all these algorithms, because it’s another place two optics quietly diverge.

The work is done by a dedicated imaging processor — often a purpose-built system-on-chip (SoC) or digital signal processor (DSP) designed for exactly this kind of per-pixel, per-frame maths. A capable chip, fed by enough memory bandwidth to shuttle a full sensor frame in and out dozens of times a second, can run richer algorithms in real time without dropping frames. A weaker or cheaper chip forces compromises — simpler routines, lighter noise reduction, a lower frame rate. Two scopes with the same sensor but different processors are not running the same software at the same quality; the hardware underneath the algorithms decides what’s even possible.

And there’s a hidden cost to everything the engine does: time. Every algorithm you add — contrast, noise reduction, edge enhancement, temporal filtering — takes milliseconds to compute, and those milliseconds stack up into latency: the small delay between what the sensor sees and what lands on your display. A gorgeous image that lags is a real problem on a moving animal; a fast image that’s noisy is hard to read. So makers are permanently balancing higher quality against lower latency, and finding that sweet spot — beautiful and responsive — takes serious engineering. It’s yet another reason two identical sensors can feel completely different in the hand.

Why Military Thermal Looks Different

Here’s a perspective that reframes everything above. If you’ve ever seen footage from a military or aerospace thermal system and wondered why it looks so much cleaner and more detailed than anything on a hunting-store shelf, the answer usually isn’t some magic sensor. It’s that those systems play by completely different rules.

A military imager is often built with resources a consumer optic can only dream of: a vast processing budget, custom silicon (ASICs) designed for one job and one job only, meticulous per-unit calibration, and expensive, sophisticated algorithms refined over decades. When cost, size and battery life aren’t the binding constraints, engineers can throw an enormous amount of processing at the image — and it shows.

A civilian hunting optic has to solve the very same problem inside a brutal set of limits. It must run for hours on a small battery, fit in your hand, shed the heat its own processor generates, and land at a price real people will actually pay. Every one of those constraints caps how much processing you can do. Seen that way, the remarkable thing isn’t that consumer thermal looks worse than military kit — it’s how astonishingly good it now looks despite running on a tiny fraction of the power, size and budget. That progress is almost entirely a story of smarter software wringing more out of less.

What Image Processing Can’t Do

It would be easy to read all this and conclude that software is magic — that a clever enough engine can rescue any sensor. It can’t, and being clear about the limits is what keeps this honest.

It can’t invent detail that was never captured. Processing works only with the data the sensor and lens delivered. If the information isn’t in the raw frame, no algorithm can conjure it — enhancement makes existing detail easier to see, it doesn’t add detail that was never there. That is the hard ceiling beneath every “AI super-resolution” claim.

It can’t fix a bad lens. If a soft or poorly-coated germanium objective hands the sensor a low-contrast, blurred scene, processing starts from a worse position and can only do so much. Garbage in, limited out — which is why the lens and sensor still matter enormously.

It can overdo it. Push contrast, sharpening or noise reduction too far and you get the tell-tale marks of over-processing: halos around warm objects, a waxy “plastic” look, crushed detail in the shadows, or smeared motion. A restrained, well-judged engine routinely beats an aggressive one. Good processing is about balance, not sliders at maximum.

The Role of Artificial Intelligence

No discussion of modern thermal processing is complete without AI — but it’s an area where marketing runs well ahead of reality, so it’s worth being precise. Machine-learning techniques are genuinely finding their way into thermal image processing, in a few real ways:

Learned noise reduction and enhancement — models trained on thousands of thermal frames can clean and sharpen an image more intelligently than fixed formulas, preserving detail a blunt filter would smear. Adaptive contrast — software that recognises the type of scene and adjusts the tone-mapping to suit. Target separation and object enhancement — algorithms that pick a warm body out of clutter and lift it, or draw a detection box around likely game. And, at the leading edge, scene recognition — classifying what’s in frame.

Where honesty matters: on a battery-powered, handheld scope, processing power and heat are hard limits, so today’s “AI” in most hunting optics is closer to smart, trained-up versions of the same enhancement and detection jobs than to full human-level scene understanding. It’s real and improving quickly, but a detection box is not identification, and no amount of AI invents detail the sensor never captured. Treat bold AI claims the way you’d treat any spec: ask what it actually does in the field.

Why Premium Thermal Optics Invest So Much in Software

Once you understand that the image is made in software, the price of premium optics starts to make sense. The expensive part of a great thermal scope increasingly isn’t only the sensor — it’s the years of engineering poured into what happens after the sensor.

A serious maker invests heavily in the things you never see on a spec sheet: a team of imaging engineers writing and refining algorithms; enormous amounts of field testing across seasons, climates and targets to tune them; firmware development and quality control; per-unit factory calibration; and ongoing updates that keep improving the product after you’ve bought it. That is a large, continuous cost — and it’s precisely why two scopes with the same sensor can sit at very different price points and be worth it.

It also reframes what you’re paying for. A budget optic and a premium one may share a sensor, but the premium one carries the accumulated tuning of a mature imaging team. You’re buying the edit, not just the ingredients.

Firmware Updates Can Improve Image Quality

Here’s something very few buyers appreciate: because the image is made in software, a thermal optic can get visibly better after you own it — with no change to the hardware at all. A firmware update can improve the contrast and enhancement algorithms, add or refine colour palettes, sharpen the NUC routine, smooth out temporal filtering, and fix stability bugs. The same sensor, behind better software, produces a better picture.

To see why that’s such a big deal, follow the life of a single scope. It ships as Version 1 — good, not perfect. A few months later an update sharpens the contrast algorithm. The next one reduces noise. The one after that refines the NUC so the image holds steadier through temperature swings. Another adds a colour palette that makes low-contrast targets pop. None of it touched a single piece of hardware — yet the owner is now looking through a visibly better scope than the one they bought. Nothing else in an optic behaves like this: the glass on a day scope is fixed forever, but a thermal image is software, and software improves.

This is genuinely underappreciated, and it should shape how you buy. A maker with a real track record of meaningful firmware updates is offering something no spec sheet can show — a product that keeps getting better after the sale. A maker who ships and forgets is selling you the best image you’ll ever get from that optic on day one. Over the years you’ll own a thermal, that difference is enormous.

Firmware turns a thermal optic into a product that can appreciate over time. It’s a genuine reason to favour makers with a track record of meaningful updates — the scope you buy may not be the scope you own in a year.

Why Thermal Images Change During the Day

You may have noticed the same scope looks sharper at 3am than at 3pm, or crisp on a cold night and washed-out after a hot day. That isn’t a fault — it’s physics, and the processing engine fighting it in real time.

Thermal contrast depends on the temperature difference between your target and its surroundings, and that difference swings constantly. In the cool of early morning a warm animal blazes against cold ground — easy. Late afternoon, after the sun has soaked heat into rocks, soil and vegetation, the background can be as warm as the animal, and contrast collapses (a phenomenon called thermal crossover). Rain and high humidity absorb infrared and cut range; wet ground and fog flatten the scene further.

Through all of this, the processing engine is continuously re-balancing — AGC re-levelling the range, contrast and dynamic-range algorithms working harder to pull a target out of a low-contrast scene. A good engine keeps the image usable in conditions where a weaker one gives up. It’s another place where two identical sensors diverge: not in the easy cold-night shot, but in the hard afternoon one.

Image Processing Myths

Four beliefs that spec-sheet shopping tends to breed — and why each one is wrong.

To take them in turn: a better sensor raises the ceiling, but a weak engine wastes it — the image is a whole-system result. A 640 sensor has more pixels than a 384, but a well-processed 384 can genuinely out-read a poorly-processed 640 (more on that in 384 vs 640). Processing is emphatically not “just sharpening” — sharpening is one algorithm among a dozen. And image quality is not fixed at purchase; firmware keeps it moving.

Why Spec Sheets Don’t Mention Image Processing

If image processing matters this much, why does no spec sheet talk about it? The answer is quietly one of the most useful insights in this whole guide — and it explains why so many buyers end up comparing the wrong things.

Some specifications are easy to print because they reduce to a single, measurable number: resolution (640×512), NETD (<20 mK), refresh rate (50 Hz), magnification (3–12×). They’re objective, comparable and marketing-friendly. Line two scopes up in a table and the numbers appear to do the talking.

Image processing is the opposite. There is no single number for “how good are your contrast algorithms,” no unit for “how well does your engine hold a moving animal in low contrast,” no agreed benchmark for tuning quality or firmware support. Software quality is real, it’s often decisive, and it is genuinely hard to quantify — so it never makes it onto the sheet.

The consequence is subtle but important. Because the easy-to-measure things get printed and the hard-to-measure things don’t, buyers naturally compare the specs that are visible rather than the ones that matter most. The spec sheet isn’t lying to you — it’s just quietly steering your attention to the part of the story that fits in a table. Simply knowing that is half the battle.

Real-World Buying Advice

If the sensor spec doesn’t tell you how good the image is, what should you actually do when choosing a thermal optic? Shift your attention from the numbers to the things that produce the picture.

Stop comparing only these: resolution, NETD and pixel pitch. They matter, but they describe potential, not performance, and they’re where marketing wants your eyes.

Start comparing these instead: the actual image (look through the optic, or study honest side-by-side field footage — your own eyes beat any spec); the maker’s reputation for image processing and tuning; independent field testing and reviews rather than lab numbers; the maturity of the brand and its imaging team; and the track record of firmware support, since that determines whether your optic improves or stagnates.

The spec sheet compares the ingredients. Real-world performance compares the finished meal. When two optics match on paper, the tie-breaker is everything the spec sheet leaves out.

The Complete System Is the Performance

The thermal sensor is the foundation of every optic, but it is not the finished product. The lens, the processing engine, the algorithms and the display all decide what you ultimately see — so when you compare thermal optics, it is the complete imaging system, not any single specification, that defines real-world performance.

And here is the deeper truth underneath all of it. The thermal industry spends enormous effort talking about sensors because sensors are easy to measure. But the greatest differences between thermal optics are increasingly found in the software — because software is difficult to measure. Once you understand that, you will never compare thermal optics the same way again.

So the next time two scopes look identical on paper, look through them instead. Watch how each one handles a hard, low-contrast afternoon. Ask who wrote the software — and whether they still improve it. That is where the real answer lives.

To go deeper on the specs this guide puts in their place: 12µm vs 17µm Pixel Pitch, What Is NETD?, 384 vs 640 Sensors, Thermal Resolution vs Lens vs Sensitivity and Detection vs Recognition vs Identification.

Frequently Asked Questions