检测图像中不均匀照明的稳健算法[仅需要检测]

Question

tesseract OCR 文本识别的最大挑战之一是图像的照明不均匀。 我需要一种算法来判断图像是否包含不均匀的照明。

测试图像

我附上了no illumination image、glare image( white-spotted image)和shadow containing image的图片。 如果我们给算法一个图像，算法应该分为两类，如

1. 没有不均匀的照明 - 我们的 no illumination image 将属于此类。
2. 照明不均匀 - 我们的 glare image( white-spotted image)、shadow containing image 将属于此类。

无照明图像 - A 类

不均匀照明图像（眩光图像（白点图像））B类

不均匀照明图像（包含图像的阴影）B 类

初始方法

1. 将色彩空间更改为 HSV

2. HSV值通道的直方图分析识别光照不均。

我们可以使用感知亮度来代替前两个步骤 渠道而不是HSV的价值渠道

1. 设置一个低阈值以获得小于低阈值的像素数

2. 设置高阈值以获取高于高阈值的像素数

3. 低像素值百分比和高像素值百分比以检测不均匀闪电情况（百分比的设置阈值也是如此）

但我找不到不均匀照明之间的很大相似之处 图片。我刚刚发现有一些像素值很低并且 一些像素在直方图分析中具有很高的价值。

基本上我的感觉是，如果将一些阈值设置为低阈值并找出有多少像素小于低阈值并​​设置一些高阈值以找出有多少像素大于该阈值。通过像素数，我们可以得出结论来检测图像中不均匀的闪电条件吗？这里我们需要最终确定两个阈值和像素数的百分比才能得出结论。

def  show_hist_v(img_path):
    img = cv2.imread(img_path)
    hsv_img = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
    h,s,v  = cv2.split(hsv_img)
    histr =cv2.calcHist(v, [0], None, [255],[0,255])
    plt.plot(histr) 
    plt.show() 
    low_threshold =np.count_nonzero(v < 50)
    high_threshold =np.count_nonzero(v >200)
    total_pixels = img.shape[0]* img.shape[1]
    percenet_low =low_threshold/total_pixels*100
    percenet_high =high_threshold/total_pixels*100
    print("Total Pixels - {}\n Pixels More than 200 - {} \n Pixels Less than 50 - {} \n Pixels percentage more than 200 - {} \n Pixel spercentage less than 50 - {} \n".format(total_pixels,high_threshold,low_threshold,percenet_low,percenet_high))

                                    
    return total_pixels,high_threshold,low_threshold,percenet_low,percenet_high

那么有人可以改进我的初始方法或提供比这种方法更好的方法来检测一般情况下图像中的不均匀照明吗？

另外，我尝试了感知亮度而不是值通道，因为值通道采用 (b,g,r) 值的最大值，所以我认为感知亮度是一个不错的选择

 def get_perceive_brightness( float_img):
    float_img = np.float64(float_img)  # unit8 will make overflow
    b, g, r = cv2.split(float_img)
    float_brightness = np.sqrt(
        (0.241 * (r ** 2)) + (0.691 * (g ** 2)) + (0.068 * (b ** 2)))
    brightness_channel = np.uint8(np.absolute(float_brightness))
    return brightness_channel

def  show_hist_v(img_path):
    img = cv2.imread(img_path)
    v = get_perceive_brightness(img)
    histr =cv2.calcHist(v, [0], None, [255],[0,255])
    plt.plot(histr) 
    plt.show() 
    low_threshold =np.count_nonzero(v < 50)
    high_threshold =np.count_nonzero(v >200)
    total_pixels = img.shape[0]* img.shape[1]
    percenet_low =low_threshold/total_pixels*100
    percenet_high =high_threshold/total_pixels*100
    print("Total Pixels - {}\n Pixels More than 200 - {} \n Pixels Less than 50 - {} \n Pixels percentage more than 200 - {} \n Pixel spercentage less than 50 - {} \n".format(total_pixels,high_threshold,low_threshold,percenet_low,percenet_high))

                                    
    return  total_pixels,high_threshold,low_threshold,percenet_low,percenet_high

感知亮度通道的直方图分析

正如 Ahmet 建议的那样。

def get_percentage_of_binary_pixels(img=None, img_path=None):
  if img is None:
    if img_path is not None:
      gray_img = cv2.imread(img_path, 0)
    else:
      return "No img or img_path"
  else:
    print(img.shape)
    if len(img.shape) > 2:
      gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    else:
      gray_img = img
  h, w = gray_img.shape
  guassian_blur = cv2.GaussianBlur(gray_img, (5, 5), 0)
  thresh_value, otsu_img = cv2.threshold(guassian_blur, 0, 255,
                                         cv2.THRESH_BINARY + cv2.THRESH_OTSU)
  cv2.imwrite("binary/{}".format(img_path.split('/')[-1]), otsu_img)
  black_pixels = np.count_nonzero(otsu_img == 0)
  # white_pixels = np.count_nonzero(otsu_img == 255)

  black_pixels_percentage = black_pixels / (h * w) * 100
  # white_pixels_percentage = white_pixels / (h * w) * 100

  return black_pixels_percentage

当我们通过大津二值化获得超过 35% 的黑色像素百分比时，我们可以检测到 80% 左右的不均匀照明图像。当光照发生在图像的小区域时，检测失败。

提前致谢

Answer 1

我建议使用除法技巧将文本与背景分开，然后仅计算背景上的统计信息。设置一些合理的阈值后，很容易为光照创建分类器。

def get_image_stats(img_path, lbl):
    img = cv2.imread(img_path)
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    blurred = cv2.GaussianBlur(gray, (25, 25), 0)
    no_text = gray * ((gray/blurred)>0.99)                     # select background only
    no_text[no_text<10] = no_text[no_text>20].mean()           # convert black pixels to mean value
    no_bright = no_text.copy()
    no_bright[no_bright>220] = no_bright[no_bright<220].mean() # disregard bright pixels

    print(lbl)
    std = no_bright.std()
    print('STD:', std)
    bright = (no_text>220).sum()
    print('Brigth pixels:', bright)
    plt.figure()
    plt.hist(no_text.reshape(-1,1), 25)
    plt.title(lbl)

    if std>25:
        print("!!! Detected uneven illumination")
    if no_text.mean()<200 and bright>8000:
        print("!!! Detected glare")

这会导致：

 good_img
STD: 11.264569863071165
Brigth pixels: 58

 glare_img
STD: 15.00149131296984
Brigth pixels: 15122
!!! Detected glare

 uneven_img
STD: 57.99510339944441
Brigth pixels: 688
!!! Detected uneven illumination

现在让我们分析直方图并应用一些常识。我们期望背景是均匀的并且具有低方差，就像“good_img”中的情况一样。如果它的方差很大，那么它的标准偏差就会很高，这是亮度不均匀的情况。在下图中，您可以看到 3 个（较小的）峰负责 3 个不同的照明区域。中间的最大峰值是将所有黑色像素设置为平均值的结果。我相信将 STD 高于 25 的图像称为“照明不均匀”情况是安全的。

当有眩光时，很容易发现大量明亮像素（见右图）。眩光图像看起来像一个很好的图像，除了热点。将明亮像素的阈值设置为 8000（总图像大小的 1.5%）应该可以很好地检测此类图像。有可能背景到处都很亮，所以如果no_text像素的平均值在200以上，那就是这样，不需要检测热点。

Answer 2

为什么不从图像中删除闪电效果？

例如：

如果我们想用pytesseract 读取输出将是' \n\f'

• 但如果我们移除闪电：

---

import cv2
import pytesseract

img = cv2.imread('img2.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
smooth = cv2.GaussianBlur(gray, (95, 95), 0)
division = cv2.divide(gray, smooth, scale=192)

• 使用pytesseract 阅读，部分输出将是：

.
.
.
Dosage & use
See package insert for compicic
information,

Instruction:
Keep all medicines out of the re.
Read the instructions carefully

Storage:
Store at temperature below 30°C.
Protect from Heat, light & moisture. BATCH NO. : 014C003
MFG. DATE - 03-2019

—— EXP. DATE : 03-2021

GENIX Distributed
AS Exclusi i :
genx PHARMA PRIVATE LIMITED Cevoka Pv 2 A ‘<
» 45-B, Kore ci
Karachi-75190, | Pakisier al Pei yaa fans
www.genixpharma.com

• 重复最后一张图片：

• 使用pytesseract 读取，部分输出将是：

.
.
.
Dosage & use
See package insert for complete prescribing
information. Rx Only

Instruction:
Keep all medicines out of the reach of children.
Read the instructions carefully before using.

Storage:

Store at temperature below 30°C. 5

Protect from Neat, light & moisture. BATCH NO, : 0140003
MFG. DATE : 03-2019
EXP. DATE : 03-2021

Manufactured by:

GENI N Exclusively Distributed by:
GENIX PHARMA PRIVATE LIMITED Ceyoka (Pvt) Ltd.

44, 45-B, Korangi Creek Road, 55, Negombe Road,
Karachi-75190, Pakistan. Peliyagoda, Snianka,

www. genixpharma.com

更新

您可以使用erode 和dilatation 方法找到被照亮的部分。

结果：

代码：

---

import cv2
import imutils
import numpy as np
from skimage import measure
from imutils import contours

img = cv2.imread('img2.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (95, 95), 0)
thresh = cv2.threshold(blurred, 200, 255, cv2.THRESH_BINARY)[1]
thresh = cv2.erode(thresh, None, iterations=2)
thresh = cv2.dilate(thresh, None, iterations=4)
labels = measure.label(thresh, neighbors=8, background=0)
mask = np.zeros(thresh.shape, dtype="uint8")
for label in np.unique(labels):
    if label == 0:
        continue
    labelMask = np.zeros(thresh.shape, dtype="uint8")
    labelMask[labels == label] = 255
    numPixels = cv2.countNonZero(labelMask)
    if numPixels > 300:
        mask = cv2.add(mask, labelMask)

    cnts = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL,
                            cv2.CHAIN_APPROX_SIMPLE)
    cnts = imutils.grab_contours(cnts)
    cnts = contours.sort_contours(cnts)[0]
    for (i, c) in enumerate(cnts):
        (x, y, w, h) = cv2.boundingRect(c)
        ((cX, cY), radius) = cv2.minEnclosingCircle(c)
        cv2.circle(img, (int(cX), int(cY)), int(radius),
                   (0, 0, 255), 3)
        cv2.putText(img, "#{}".format(i + 1), (x, y - 15),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.45, (0, 0, 255), 2)
    cv2.imshow("Image", img)
    cv2.waitKey(0)

虽然我只测试了第二张图片。您可能需要更改其他图像的参数。

Answer 3

这是 ImageMagick 中的快速解决方案。但它可以很容易地在 Python/OpenCV 中实现，如下图所示。

使用除法归一化。

• 读取输入
• 可选择转换为灰度
• 复制图像并对其进行模糊处理
• 将模糊图像除以原始图像
• 保存结果

输入：

convert 8W0bp.jpg \( +clone -blur 0x13 \) +swap -compose divide -composite x1.png

convert ob87W.jpg \( +clone -blur 0x13 \) +swap -compose divide -composite x2.png

convert HLJuA.jpg \( +clone -blur 0x13 \) +swap -compose divide -composite x3.png

结果：

在 Python/OpenCV 中：

import cv2
import numpy as np
import skimage.filters as filters

# read the image
img = cv2.imread('8W0bp.jpg')
#img = cv2.imread('ob87W.jpg')
#img = cv2.imread('HLJuA.jpg')

# convert to gray
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)

# blur
smooth = cv2.GaussianBlur(gray, (33,33), 0)

# divide gray by morphology image
division = cv2.divide(gray, smooth, scale=255)

# sharpen using unsharp masking
sharp = filters.unsharp_mask(division, radius=1.5, amount=2.5, multichannel=False, preserve_range=False)
sharp = (255*sharp).clip(0,255).astype(np.uint8)

# save results
cv2.imwrite('8W0bp_division.jpg',division)
cv2.imwrite('8W0bp_division_sharp.jpg',sharp)
#cv2.imwrite('ob87W_division.jpg',division)
#cv2.imwrite('ob87W_division_sharp.jpg',sharp)
#cv2.imwrite('HLJuA_division.jpg',division)
#cv2.imwrite('HLJuA_division_sharp.jpg',sharp)

# show results
cv2.imshow('smooth', smooth)  
cv2.imshow('division', division)  
cv2.imshow('sharp', sharp)  
cv2.waitKey(0)
cv2.destroyAllWindows()

结果：

Answer 4

这是我的管道：

%matplotlib inline
import numpy as np
import cv2
from matplotlib import pyplot as plt
from scipy.signal import find_peaks 

我使用函数：

def get_perceived_brightness( float_img):
    float_img = np.float64(float_img)  # unit8 will make overflow
    b, g, r = cv2.split(float_img)
    float_brightness = np.sqrt((0.241 * (r ** 2)) + (0.691 * (g ** 2)) + (0.068 * (b ** 2)))
    brightness_channel = np.uint8(np.absolute(float_brightness))
    return brightness_channel
    
# from: https://stackoverflow.com/questions/46300577/find-locale-minimum-in-histogram-1d-array-python
def smooth(x,window_len=11,window='hanning'):
    if x.ndim != 1:
        raise ValueError("smooth only accepts 1 dimension arrays.")

    if x.size < window_len:
        raise ValueError("Input vector needs to be bigger than window size.")

    if window_len<3:
        return x

    if not window in ['flat', 'hanning', 'hamming', 'bartlett', 'blackman']:
        raise ValueError("Window is on of 'flat', 'hanning', 'hamming', 'bartlett', 'blackman'")

    s=np.r_[x[window_len-1:0:-1],x,x[-2:-window_len-1:-1]]

    if window == 'flat': #moving average
        w=np.ones(window_len,'d')
    else:
        w=eval('np.'+window+'(window_len)')

    y=np.convolve(w/w.sum(),s,mode='valid')
    return y
    

我加载图片

image_file_name = 'im3.jpg'
image = cv2.imread(image_file_name)

# image category
category = 0

# gray convertion
image_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

height = image.shape[0]
width = image.shape[1]

第一次测试。图片有没有大的白点？

# First test. Does the image have any big white spots?
saturation_thresh = 250
raw_saturation_region = cv2.threshold(image_gray, saturation_thresh, 255,  cv2.THRESH_BINARY)[1]
num_raw_saturation_regions, raw_saturation_regions,stats, _ = cv2.connectedComponentsWithStats(raw_saturation_region)

# index 0 is the background -> to remove
area_raw_saturation_regions = stats[1:,4]

min_area_bad_spot = 1000 # this can be calculated as percentage of the image area
if (np.max(area_raw_saturation_regions) > min_area_bad_spot):
    category = 2 # there is at least one spot

图像法线的结果：

带有斑点的图像的结果：

带有阴影的图像的结果：

如果图像通过第一个测试，我会处理第二个测试。图像暗吗？

# Second test. Is the image dark?   
min_mean_intensity = 60

if category == 0 :    
    mean_intensity = np.mean(image_gray)

    if (mean_intensity < min_mean_intensity):
        category = 3 # dark image
        

如果图像也通过了第二个测试，我将处理第三个测试。图像照度均匀吗？

window_len = 15 # odd number
delay = int((window_len-1)/2)  # delay is the shift introduced from the smoothing. It's half window_len

# for example if the window_len is 15, the delay is 7
# infact hist.shape = 256 and smooted_hist.shape = 270 (= 256 + 2*delay)

if category == 0 :  
    perceived_brightness = get_perceived_brightness(image)
    hist,bins = np.histogram(perceived_brightness.ravel(),256,[0,256])

    # smoothed_hist is shifted from the original one    
    smoothed_hist = smooth(hist,window_len)
    
    # smoothed histogram syncronized with the original histogram
    sync_smoothed_hist = smoothed_hist[delay:-delay]    
    
    # if number the peaks with:
    #    20<bin<250
    #    prominance >= mean histogram value
    # the image could have shadows (but it could have also a background with some colors)
    mean_hist = int(height*width / 256)

    peaks, _ = find_peaks(sync_smoothed_hist, prominence=mean_hist)
    
    selected_peaks = peaks[(peaks > 20) & (peaks < 250)]
    
    if (selected_peaks.size>1) :
        category = 4 # there are shadows

图像法线的直方图：

带有斑点的图像的直方图：

带有阴影的图像的直方图：

如果图片通过了所有测试，那就是正常

# all tests are passed. The image is ok
if (category == 0) :
    category=1 # the image is ok