Brightness Histogram and Equalization

 Brightness histogram

Brightness histogram is a graph showing the distribution of pixel values in an image. The horizontal axis is the pixel value (0 to 255), and the vertical axis is the number of pixels with the corresponding pixel value.

Let's create a code that implements a brightness histogram. As you can see from the code below, matplolib must be installed in advance.

First let's see one image named big-one.jpg

import sys
import numpy as np
import cv2
from matplotlib import pyplot as plt, gridspec

image = 'C:/lsh/study/image/big-one.jpg'

im = cv2.imread(image)
B = im[:,:, 0]
G = im[:,:, 1]
R = im[:,:, 2]
cv2.imshow("dark", im)
cv2.waitKey(0)
cv2.destroyAllWindows()

This is the big-one.jpg image.

Now let's implement the code that outputs the histogram of the image in earnest. The code is very simple. We will split the RGB image into R,G,B channels and then look at the pixel values per channel. There is a function called histogram in numpy that creates a histogram. Using this function, the histogram data of R, G, and B channels can be easily obtained.

The following is the usage of the histogram function taken from the numpy manual. Important parameters are bins and range. Thr "range" corresponds to the area on the x-axis. Since pixel values range from 0 to 255, range = (0,255) is specified. And bins determines how many range values are divided into graphs. I want to see the values for all pixel values (0 to 255). So this value will be 256.

numpy.histogram(a, bins=10, range=None, normed=None, weights=None, density=None)
[source]
¶
Compute the histogram of a dataset.
Parameters
aarray_like
Input data. The histogram is computed over the flattened array.
binsint or sequence of scalars or str, optional
If bins is an int, it defines the number of equal-width bins in the given range (10, by default). If bins is a sequence, it defines a monotonically increasing array of bin edges, including the rightmost edge, allowing for non-uniform bin widths.
New in version 1.11.0.
If bins is a string, it defines the method used to calculate the optimal bin width, as defined by histogram_bin_edges.
range(float, float), optional
The lower and upper range of the bins. If not provided, range is simply (a.min(), a.max()). Values outside the range are ignored. The first element of the range must be less than or equal to the second. range affects the automatic bin computation as well. While bin width is computed to be optimal based on the actual data within range, the bin count will fill the entire range including portions containing no data.
normedbool, optional
Deprecated since version 1.6.0.
This is equivalent to the density argument, but produces incorrect results for unequal bin widths. It should not be used.
Changed in version 1.15.0: DeprecationWarnings are actually emitted.
weightsarray_like, optional
An array of weights, of the same shape as a. Each value in a only contributes its associated weight towards the bin count (instead of 1). If density is True, the weights are normalized, so that the integral of the density over the range remains 1.
densitybool, optional
If False, the result will contain the number of samples in each bin. If True, the result is the value of the probability density function at the bin, normalized such that the integral over the range is 1. Note that the sum of the histogram values will not be equal to 1 unless bins of unity width are chosen; it is not a probability mass function. Overrides  the normed keyword if given.

Returns

histarray

The values of the histogram. See density and weights for a description of the possible semantics.

bin_edgesarray of dtype float

Return the bin edges (length(hist)+1)

plt.figure()
plt.title(" Histogram")
plt.xlabel("pixel value")
plt.ylabel("pixel num")
plt.xlim([0.0, 255])  # <- named arguments do not work here

colors = ("blue", "red", "green")
histogram_B, bin_edges_B = np.histogram(B, bins=256, range=(0, 255))
histogram_G, bin_edges_G = np.histogram(G, bins=256, range=(0, 255))
histogram_R, bin_edges_R = np.histogram(R, bins=256, range=(0, 255))

plt.plot(bin_edges_B[0:-1], histogram_B, color = "blue")  # <- or here
plt.plot(bin_edges_G[0:-1], histogram_G, color = "green")  # <- or here
plt.plot(bin_edges_R[0:-1], histogram_R, color = "red")  # <- or here
plt.show()

This is the result of above code. 

It can be seen that the pixel value is largely divided into a very dark area and a very bright area for R channel(pixels close to 255).

If the histogram is divided at both ends like this, it is difficult to adjust the histogram. If you increase the brightness, areas close to 255 (the snow and part of the sky in the picture) are likely to lose all detail. Conversely, if you lower the brightness, all the detail in the dark areas is lost and it is more likely to turn black.

Histogram Equalization

Much of the content from now on was referred to in 

• Changing the contrast and brightness of an image!
• Histograms - 2: Histogram Equalization

Theory

Consider an image whose pixel values are confined to some specific range of values only. For eg, brighter image will have all pixels confined to high values. But a good image will have pixels from all regions of the image. So you need to stretch this histogram to either ends (as given in below image, from wikipedia) and that is what Histogram Equalization does (in simple words). This normally improves the contrast of the image.

I would recommend you to read the wikipedia page on Histogram Equalization for more details about it. It has a very good explanation with worked out examples, so that you would understand almost everything after reading that. Instead, here we will see its Numpy implementation. After that, we will see OpenCV function.

I'm going to open the wiki.jpg file.

import sys
import numpy as np
import cv2
from matplotlib import pyplot as plt, gridspec

image = 'C:/lsh/study/image/wiki.jpg'

im = cv2.imread(image)
B = im[:,:, 0]
G = im[:,:, 1]
R = im[:,:, 2]
cv2.imshow("dark", im)
cv2.waitKey(0)
cv2.destroyAllWindows()

This image is not sharp due to low contrast. 

I will print the histogram using the method introduced above. This time, the histogram of the RGB channel will be output as three graphs independently. The reason is that the RGB channel values of the images used for testing are all the same, so the graphs appear overlapping. This is what happens when you convert a black and white image (grayscale) to a color image.

plt.figure(figsize=(15,5))
grid = gridspec.GridSpec(1,3)
ax0 = plt.subplot(grid[0])
ax1 = plt.subplot(grid[1])
ax2 = plt.subplot(grid[2])

plt.title(" Histogram")
plt.xlabel("pixel value")
plt.ylabel("pixel num")
plt.xlim([0.0, 255])  # <- named arguments do not work here

colors = ("blue", "red", "green")
histogram_B, bin_edges_B = np.histogram(B, bins=256, range=(0, 255))
histogram_G, bin_edges_G = np.histogram(G, bins=256, range=(0, 255))
histogram_R, bin_edges_R = np.histogram(R, bins=256, range=(0, 255))

ax0.plot(bin_edges_B[0:-1], histogram_B, color = "blue")
ax1.plot(bin_edges_G[0:-1], histogram_G, color = "green")
ax2.plot(bin_edges_B[0:-1], histogram_R, color = "red")

plt.show()

As can be seen from the histogram, most of the pixel values are concentrated around 150.

You can see histogram lies in brighter region. We need the full spectrum. For that, we need a transformation function which maps the input pixels in brighter region to output pixels in full region. That is what histogram equalization does.

Now we find the minimum histogram value (excluding 0) and apply the histogram equalization equation as given in wiki page. But I have used here, the masked array concept array from Numpy. For masked array, all operations are performed on non-masked elements. You can read more about it from Numpy docs on masked arrays.

hist, bins = np.histogram(im, bins=256, range=(0, 255)) #I'm going to make histogram for BGR at once

cdf = hist.cumsum()

cdf_m = np.ma.masked_equal(cdf,0)
cdf_m = (cdf_m - cdf_m.min())*255/(cdf_m.max()-cdf_m.min())
cdf = np.ma.filled(cdf_m,0).astype('uint8')
im2 = cdf[im]

new_hist, new_bins = np.histogram(im2, bins=256, range=(0, 255)) #I'm going to make histogram for BGR at once


plt.figure(figsize=(15,5))
grid = gridspec.GridSpec(1,2)
ax0 = plt.subplot(grid[0])
ax1 = plt.subplot(grid[1])
ax1.imshow(im2)
ax1.plot(bins[0:-1], new_hist, color = "black")

plt.show()

Now the result is.

Another important feature is that, even if the image was a darker image (instead of a brighter one we used), after equalization we will get almost the same image as we got. As a result, this is used as a "reference tool" to make all images with same lighting conditions. This is useful in many cases. For example, in face recognition, before training the face data, the images of faces are histogram equalized to make them all with same lighting conditions.

Histogram equalization is good when histogram of the image is confined to a particular region. It won't work good in places where there is large intensity variations where histogram covers a large region, ie both bright and dark pixels are present. Please check the SOF links in Additional Resources.

CLAHE (Contrast Limited Adaptive Histogram Equalization)

The first histogram equalization we just saw, considers the global contrast of the image. In many cases, it is not a good idea. For example, below image shows an input image and its result after global histogram equalization.

This time, we will use the tsukuba_L.png image. You can easily find this image by doing a Google search. And instead of the histogram, let's look at the original image and the image with the histogram adjusted.

hist, bins = np.histogram(im, bins=256, range=(0, 255)) #I'm going to make histogram for BGR at once

cdf = hist.cumsum()

cdf_m = np.ma.masked_equal(cdf,0)
cdf_m = (cdf_m - cdf_m.min())*255/(cdf_m.max()-cdf_m.min())
cdf = np.ma.filled(cdf_m,0).astype('uint8')
im2 = cdf[im]

new_hist, new_bins = np.histogram(im2, bins=256, range=(0, 255)) #I'm going to make histogram for BGR at once


plt.figure(figsize=(15,5))
grid = gridspec.GridSpec(1,2)
ax0 = plt.subplot(grid[0])
ax1 = plt.subplot(grid[1])
ax0.imshow(cv2.cvtColor(im, cv2.COLOR_BGR2RGB))
ax1.imshow(cv2.cvtColor(im2, cv2.COLOR_BGR2RGB))

plt.show()

Below image shows an input image and its result after global histogram equalization. It is true that the background contrast has improved after histogram equalization. But compare the face of statue in both images. We lost most of the information there due to over-brightness. It is because its histogram is not confined to a particular region as we saw in previous cases (Try to plot histogram of input image, you will get more intuition).

So to solve this problem, adaptive histogram equalization is used. In this, image is divided into small blocks called "tiles" (tileSize is 8x8 by default in OpenCV). Then each of these blocks are histogram equalized as usual. So in a small area, histogram would confine to a small region (unless there is noise). If noise is there, it will be amplified. To avoid this, contrast limiting is applied. If any histogram bin is above the specified contrast limit (by default 40 in OpenCV), those pixels are clipped and distributed uniformly to other bins before applying histogram equalization. After equalization, to remove artifacts in tile borders, bilinear interpolation is applied.

Below code snippet shows how to apply CLAHE in OpenCV:

import sys
import numpy as np
import cv2
from matplotlib import pyplot as plt, gridspec

image = 'C:/lsh/study/image/tsukuba_L.png'

im = cv2.imread(image)

hsv = cv2.cvtColor(im, cv2.COLOR_BGR2HSV)
hsv_planes = cv2.split(hsv)
clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
hsv_planes[2] = clahe.apply(hsv_planes[2])
hsv = cv2.merge(hsv_planes)
rgb_image = cv2.cvtColor(hsv, cv2.COLOR_HSV2RGB)    


plt.figure(figsize=(15,5))
grid = gridspec.GridSpec(1,2)
ax0 = plt.subplot(grid[0])
ax1 = plt.subplot(grid[1])
ax0.imshow(cv2.cvtColor(im, cv2.COLOR_BGR2RGB))
ax1.imshow(rgb_image)

plt.show()

Be careful : There is no process of changing to HSV color in the article on the OpenCV homepage. It is possible for gray images, but color images need to be converted to HSV as in the example in this article, then work on the V channel, and then back to BGR or RGB color.

The resulting image looks much better. In particular, the statue area is expressed much better.

Wrapping up

Histogram adjustment using CLAHE is particularly effective when the object of the AI model's inference image is a human face. If the boundaries of the eyes, nose, mouth, etc. are blurred,  the recognition rate will be much better if the outline is clearly transformed using CLAHE and then passed to the AI model.

<normal image and enhanced image using CLAHE>