基于圖像處理的車型識(shí)別

基于圖像處理的車型識(shí)別,基于,圖像,圖象,處理,車型,識(shí)別,辨認(rèn) An Efficient Vehicle Model Recognition MethodHuihuaYangGuangxi Experiment Center of Information Science, Guilin, China Email: yanghuihua@tsinghua.edu.cnLei Zhai, Lingqiao Li, Zhenbing Liu, Yichen Luo, Yong WangSchool of Electronic Engineering and Automation/Guilin University of Electronic Technology, Guilin, China Email: keylei203@gmail.com, zbliu@guet.edu.cn, 54pe@163.com, louisluo@guet.edu.cn, wang@guet.edu.cnHaiyan Lai, Ming GuanGuangxi Communications Investment Group CO., LTD, Guilin, China 13517810019@163.com, gm9099@sina.comAbstract—An efficient vehicle model recognition method based on Adaptive Harris corner detector is presented in this paper. First, the vehicle radiator grid is selected as ROI and Harris corner detection is used to detect corner as vehicle model features, to solve a problem of inconsistencies in the number of corner between different models or the same model in different environment. Second, an adaptive threshold function is constructed to control the number of corner replacing a fixed threshold, ensuring that the image is always able to produce a certain number of strong corners. Third, a parallel scheme is designed to accelerate the vehicle recognition algorithm via GPU/CPU heterogeneous computing model to meet real-time requirement, which includes parallelization of algorithm and parallelization of process. The experiments on 1096 big truck images of 12 vehicle models obtain the recognition accuracy rate of 99.5%, and achieve 58x speedup on average by a platform with Intel Core i5 2400 and NVIDIA C2075. The results show that our proposed method can meet the requirements of practical application.Index Terms—vehicle model recognition; ROI positioning; adaptive Harris Algorithm; GPU/CPU collaborative computingI. INTRODUCTIONVehicle model recognition is an interesting and difficult subject of intelligent transportation study. In [1] and [2], features of corner point are utilized to achieve the coarse classification of cars, buses, heavy goods vehicles (HGVs) etc. In [3], an interesting approach for vehicle model recognition from frontal view vehicle images is presented, whose recognition accuracy achieves 93%. The vehicle manufacture and model were treated as a single class and recognized simultaneously, but no results for recognition speed were reported. In [4] and [5], a comparative knowledge acquisition system were introduced, consisting of several object recognition modules which represent a car image viewed from the rear, such as a window, tail lights, and so on, based on color recognition. This approach has the drawback ofbeing sensitive to lighting condition. In [6], a vehicle model recognition method was presented which was extracted of textural features of the radiator grille using based gray level co-occurrence matrix (GLCM), but ROI positioning is difficult in an outdoor environment. In [7], scale invariant feature transform (SIFT) features is proved to be suitable for vehicle manufacturer and model recognition, but it does not have real-time performance.In summary, the above model identification methods still can’t be a good solution to the actual vehicle recognition task in an outdoor environment. There are two main reasons:1. ROI Positioning and feature extraction are difficult in the complex outdoor environment.2. Recognition algorithm can’t meet the real-time requirement for practical applications.In the same time, GPU/CPU collaborative computing has become more and more important in data processing of computing-intensive tasks, and attracts the attention of many application developers in recent years [9][11,12]. This is because GPU can offer extensive resources even for non-visual, general-purpose computations: massive parallelism, high memory bandwidth, and general purpose instruction sets. GPU/CPU collaborative computing uses CPU to process sequence recognition tasks and uses GPU to process a large number of repetitive repeatabilitycalculations to improve the key performance of applications. GPU/CPU collaborative computing is widely used in the field of image processing [10]. To the best of our knowledge, there is no report about the application of this accelerated technology in the vehicle model recognition fields.This paper present a new vehicle model recognition based adaptive Harris corner detector. This is a part of the “JT-G green channel inspection system”, which uses vehicle model recognition technology based on machine vision to achieve the cab’s automatic detection and safe avoidance. First, radiator grid is positioned as ROI based on logo. Then, using the adaptive Harris corner detector to detect ROI’s corner, an adaptive threshold function is? 2013 ACADEMY PUBLISHER doi:10.4304/jsw.8.8.1952-1959? 2013 ACADEMY PUBLISHERconstructed to control the number of corner replace a fixed threshold. This method can ensure the image always produce a certain number of strong corner, avoiding dramatic changes of corner number between the different models, or the same model in different environment, thus it can greatly improve the recognition accuracy and robustness of the recognition.Output: The binary image after coarse location.Step: a. Suppress horizontal texture using operator [1,-1].Dx (i, j) ? g(i, j) ? g(i ? 1, j)b. Suppress vertical texture using operator[1,-1]T.D(i, j) ? Dx (i, j) ? Dx (i, j+1)c. Suppress noise.(1)(2)Besides, a parallelization scheme for vehicle model recognition is designed, including the parallelization of adaptive Harris algorithm and the parallelization of?1f (i, j) ? ??0D(i, j) ? 0D(i, j) ? 0workflow, to ensure that the algorithm has a fast response time and meet real-time requirement.i ? r j ? r sum(i, j) ? ?? f (i, j)i ?r j ?r (3)II. ROI POSITIONINGSelection and robust positioning of ROI in complexity field environment is difficult. In [6], the vehicle radiator grid as ROI completes a good vehicle recognition effect.?255pixel (i, j) ? ?? 0T = r ? r / 2 ? 1sum(i, j) ? T sum(i, j) ≤ TThis paper also uses this method, but adds logo’s information in order to increase ROI’s distinguishing degree, as shown in Fig. 1.Figure 1. ROI selection of vehicle model recognitionThe ROI is positioned based on logo’s position. The position of the number plate region is steady-going and fixed in size. If the number plate can be fast and coarse detected, it can also be obtained the vehicle logo position in the vertical direction. The relationship between the number plate and the vehicle logo is as shown in Fig. 2. General logo is always at right above the license plate’ position, we only need to detect logo in the vertical direction or position and the large white rectangle image is as the input image.Figure 2. Through the license plate to determine the approximate location of logoTo position ROI, logo vertical position is needed to determine. Algorithm 1 describes the process of vertical position location for vehicle logo.Algorithm1: Location for logo’s vertical position.Input: Grayscale image of extracted based on plate.Where r is kernel’s size, T is adaptive threshold.d. The edge extraction: detect edge using Canny operator, the output is pixel*.e. Obtaining the row that its pixel values is maximum as the horizontal axis.ROI positioning for logo vertical position is shown in Fig. 3. Where h is plate’s height, w is plate’s width, y is vertical position of logo, n is an empirical parameter.(a) Coarse location based on plate(b) Location of logo vertical position(c) Location of ROI based on logo Figure 3. Location of ROI? 2013 ACADEMY PUBLISHERIII. ADAPTIVE HARRIS ALGORITHMA. HarrisCorner is known as interesting points in the image. They are discrete, reliable and meaningful. Feature-corner is shown to perform with good consistency on natural imagery, this paper selects corner as characteristic of vehicle’s model.There are many corner detectors, such as Harris, Fast, SIFT, SURF and so on. Fig. 4 is a corner-image detected by Harris, SURF and SIFT respectively.(a) Harris (b) SURF (c) SIFT Figure 4. Corner-image detected by three detectorsAs shown in Fig.4, Harris corner detector has the best results in the view of the positioning accuracy of corner. Since Harris corner detector is based on first order Hessian matrix, so it has higher detection accuracy. SIFT or SURF is based on High-dimensional scale space, and has higher dimensional feature vectors, robustness better than Harris. Because only information of corner’s position is used in the process of matching, we selected Harris corner detector as this paper’s detector.B. Harris corner detectorHarris corner detection is a classic corner detection algorithm proposed by Harris C and Davis L S.A in 1988. It is a signal-based feature-corner extraction operator, assuming window W as processed image, the minute displacement (u, v) moving in any direction, then the gradation change amount can be defined as (4):E(u, v) ? ? w(x, y)[I (x ? u, y ? v) ? I (x, y)]2x, y? Au2 ? 2Cuv ? Bv2C. Defects of classic Harris for vehicle model recognitionThe corner response value of R, R is more than a fixed threshold value T. It is considered that point is the corner point, otherwise it is not. Fixed threshold will have some problems in the model recognition.1. The differences of corner points’ counts between different vehicle model lead to mismatch. As shown in Fig. 6, vehicle A has little corner points, vehicle B has many corner points, A and C with the same models, A and B with the different models. The counts of corner points matched between A and B successfully more than those between A and C in this case, so we concluded that A and B have the same model, vehicle model recognition for A is failure.(a) A and B have 22 similar corner points(b) A and C have 20 similar corner points? A M ? ?? ?uC??v?M ?u v?T (4) Figure 6. Differences of corner points between different vehicle model lead to mismatchWhere ?C B? is a symmetrical autocorrelation matrix,A ? X 2 ? w , function.B ? Y 2 ? w , C ? ( XY ) ? w , w(x,y) is window 2. The differences of corner points’ counts in differentillumination condition lead to mismatch. As shown in Fig. 7, vehicle D and F have the same models in different illumination condition, vehicle D and E have differentHarris’s feature is defined as the maximum value of the local, and its response function is:CRF ? ?M ? K (traceM )2models in well-lighting condition. Because Harris corner detector’s adaptive character toward lighting is not well, the corner points’ count in well-lighting condition is large?M ? ?1?2tr(M ) ? ?1 ? ?2(5) more than that in poor lighting condition. The counts of corner points matched between D and E is successfully more than those between D and F in this case. So weAs is a constant set by experience, K is generallybetween 0.04-0.06. If a point’s CRF is more threshold T, is a corner. Harris corner detection operator need to set the parameters K and T. The parameter selection arbitrariness will affect the final test results significantly. Algorithm process is as shown in Fig. 5.concluded that D and E have the same model, vehicle model recognition for D is failure.Figure 5. Processing steps for Harris corner detector? 2013 ACADEMY PUBLISHERw h0?(a) D and E have 14 similar corner points(b) D and F have 11 similar corner pointsFigure 7. The differences of corner points in different illumination condition model lead to mismatchAs shown in Fig.6 and 7, the classic Harris corner detector algorithm for vehicle model recognition effect is not good. The key reason is that the number of corner points is difficult to control. There is a large gap of the number of corner points between detected in the different models or in different environments, so it often leads a mismatch.D. Adaptive Harris AlgorithmCorner points’ count caused by the fixed threshold number of inconsistencies affect the recognition accuracy of the models. Corner threshold should be related to the overall distribution of the response value and the change range. So we take the mean of responses and the variation range weighted sum as a final adaptive threshold, Specific (6):threshold ? 1 (?? f (x, y)) ? ? *[max( f (x, y)) ? min( f (x, y))]w ? h x?1 y ?1(a) 60 corner points by hand(b) Fixed threshold(c) Adaptive thresholdFigure 8. Comparison-image using fixed threshold and adaptive thresholdAfter the calculation of the (6), corner points’ count in different models tends to be consistent in different environment, but not entirely. Sorting corner’s response value is executed in order to improve accuracy:1. Sorting corner by their response value. Strong-corner front, weak-corner behind.2. Setting a fixed parameter n, it indicates the desired output number of corner points . It is usually less than the expected minimum number of corner.Adaptive Harris algorithm process is shown in Fig. 9.f (x, y) ? ?r(x, y) r(x, y) ? t? r(x, y) ?? tr(x, y) ? ?(M )tr(M )2 (6)Where r(x, y) represents coordinate (x, y)’s response values. tpresets a small empirical threshold.The adaptive threshold is decided by mean value and change range of candidate corner, which reflects variation in the intensity of the image as a whole. If a certain vehicle model has single texture detail or in poor lighting conditions, the gradient of the image is small, the relatively response value is low, the adaptive threshold is also reduced, and still be able to ensure that the angle of the output of a certain number of points. In contrast, the adaptive threshold is also high, and it can limited excessive corner.As shown in Fig.8, figure corner points’ count is 60 points. Fig. 8(b) is corner-image using fixed threshold and Fig.8(c) is corner-image using (3). We can see that adaptive Harris threshold has a higher accuracy. The adaptive threshold can avoid the setting of the threshold value, ensure the detection accuracy at the same time.Adaptive Harris algorithm avoids models and differences between model or environment affect recognition accuracy. As shown in Fig. 10 ( the same vehicles as Fig. 6 ), vehicle A, B and C all have 15 corner, vehicle A and B have 5 corner points and A and C have 13 corner points, we conclude A with C are more similar than A with B.Figure 9. Processing steps of Adaptive Harris? 2013 ACADEMY PUBLISHER(a) D and E have 14 similar corner points(b) D and F have 11 similar corner points Figure 10. Model recognition for vehicle DAdaptive threshold can be guaranteed a certain amount of corner points to some extent by setting t and ? . The number of corner points impacts on the recognition accuracy of vehicle model. Few feature-corner generates a false match easily, more feature-corner increases amount of calculation. The perfect quantity is sufficient to reflect the characteristics of the models, but also to avoid the “false corner”. Fig. 11 is statistical analysis about vehicle model accuracy under different number of corner points detected by Harris, SURF and SIFT algorithm respectly. The results show that when the number of corner points in the range of 100-150 (t = 15, ? = 0.05) the recognition accuracy is relatively stable. When the corner points to 130, the Harris algorithm models achieve the highest recognition accuracy rate up to 99%.As shown in Fig. 12, using database matching to calculate the similarity of models’ image and the advantage is that the method is simple, maintain and upgrade easily. Its disadvantage is that the calculation is extraordinarily time consumption in the case of big database and can’t meet the real-time requirements.Figure 12. Processes of vehicle model recognition algorithmA. Parallelization of algorithmParallelization of algorithm includes parallel adaptive Harris algorithm and Max-correlation algorithm. These two algorithms are both compute-intensive, very suitable for parallel acceleration. Table I analysis of the processing flow of these two algorithms carefully, irrelevant loop and matrix operations are transmitted to GPU processing, transmission of signals and logic processing by CPU.Sample images are compressed to the size of 200? 200 in this paper. Every calculation of corner detectionand corner matching volume is not too large. In addition, the communication between the GPU and CPU overhead is large, and the speedup of parallel algorithm is not very high.TABLE I. PARALLELIZATION OF ADAPTIVE HARRIS11001050Harris SIFT SURF100095090085080050 100 150 200 250Feature-corner amoutFigure 11. Effect of the number of corner points on vehicle recognition accuracyIV. GPU/CPU COLLABORATIVE COMPUTINGThe proposed method for vehicle model recognition includes ROI positioning, corner detection and corner matching. The corner detection uses adaptive Harris algorithm, and corner matching uses the maximum correlation algorithm (Max-correlation).B. Parallelization of workflowFor the entire vehicle recognition method, due to corner detection in the whole process is called only once, the contribution of the acceleration to the entire model recognition is limited. Max-correlation algorithm is called every matching process, but the amount of calculation of the algorithm itself is small. So it is not the bottleneck of the whole method efficiency. Most of the recognition time consuming in the processing of data transmission, acceleration efficiency is not very high.Algorithm Step Parallelizable (Y/N)Adaptive Harris1.Grayscale Y2.Gradient Y3.Gaussian blur Y4 Correlation matrix Y5.Response Y6.Non-maximum suppression Y7.Adaptive threshold N9.Sorting corner NMax-correlation8. Calculating a correlation matrix Y9.Search match corner Y10. Calculating number of match corner Y1091/1096 Thecorrect amount? 2013 ACADEMY PUBLISHERImage of current vehicleDetect corner using Adaptive HarrisSince every corner matching process is independent, the matching of the different vehicle models can be parallelized. We select NVIDIA CUDA platform, used optimization techniques are as follows:1. Loading all features of datasets first, some constant as Gauss coefficients were loaded to constant registers of GPU.2. Setting the number of feature database as thread blocks, setting the number of corner as the number of threads in a thread block.Parallelization of workflow is realized in Fig. 13.Figure 13. Frame of parallel model for vehicle model recognitionV. EXPERIMENT RESULTS AND ANALYSISA. Data set description and experimental environmentThe datasets come from “JT-G green channel inspection system” on Guangxi “Quan-Yellow” high-speed toll station. Datasets include 6 categories logo, 12 vehicle models and 1096 front and side images with the speed of 3-20 km/h speed passing the equipment. Affected by the sensitivity of the sensor and the vehicle’s uniform speed, vehicle distance and angle may have a slight deviation. The samples also contain different lighting conditions such as sunny, cloudy, and evening. Experimental environment as following:1. PC: Intel Core i5-2400@3.10GHz x4, 4GB memory, win7 sp1 x64 operating system.2. GPU: NVIDIA Tesla C2075, CUDA v4.2.Figure 14. Green channel inspection system on the freewayB. Recognition AccuracyThe classic image “Lena” for corner detection is selected in order to verify the accuracy of the adaptive Harris algorithm based on GPU/CPU collaborative computing. The comparative results are shown in Fig. 15.We can see that the two corner-image are almost the same, which proves the accuracy of the algorithm. It is noteworthy that, these two corner-images are not same completely. That is because the inconsistencies of the floating calculation accuracy between CPU and GPU. It leads to the result of the calculation having a small deviation.(a) corner-image of CPU-Version algorithm(b) corner-image of GPU/CPU-Version algorithmFigure 15. Effect comparison of CPU-Version and GPU/CPU-version of the algorithmVehicle model recognition has many feature-extraction methods, this paper compared GLCM, PHash, DCT, SURF, SIFT and Harris, the results are shown in Table II.………………Matching Matching Matching ……… Matching corner N1 corner N2