Similar & Duplicate Image Detection
Implement a highly efficient Average Hash (aHash) algorithm combined with Disjoint Set Union (DSU) in C++ to group thousands of images without Out-Of-Memory (OOM) crashes.
Step 1: Architecture & Algorithm
Why aHash and file paths?
aHash + Hamming Distance
Average Hashing (aHash) shrinks an image to 8x8 pixels, converts it to grayscale, and compares each pixel to the mean to generate a 64-bit integer. We then use __builtin_popcountll to rapidly compute the Hamming distance.
Anti-OOM Strategy
Passing an Array<Bitmap> of 100+ images from Kotlin to JNI will quickly exhaust memory. Instead, we pass an Array<String> of file paths. C++ reads, hashes, and releases them sequentially.
Step 2: Native C++ Implementation
Hash extraction and Disjoint Set Union grouping
Step 3: Kotlin Integration
Map the native IntArray back into grouped lists of strings.
packagecom.wefamily.papa.utilsobjectOpenCVUtils{init{ System.loadLibrary("image_matcher") }/** * 底层 JNI 接口 * @param imagePaths 图片的绝对路径数组 * @param threshold 汉明距离阈值 (建议值: 0-10,越小要求越相似) * @return 返回与数组长度一致的 IntArray,值为分组的 Group ID */private external fungroupSimilarImages( imagePaths:Array<String>, threshold:Int):IntArray/** * 业务层调用的便捷封装 */funfindDuplicates(paths:List<String>, threshold:Int=5):List<List<String>> {if(paths.isEmpty())returnemptyList()// 1. 传入底层计算分组 IDvalgroupIds = groupSimilarImages(paths.toTypedArray(), threshold)// 2. 使用 Kotlin 高阶函数进行数据组合与过滤returnpaths.zip(groupIds.toList()) .groupBy { it.second }// 按 Group ID 分组.values .map { group -> group.map { it.first } }// 提取出原路径.filter { it.size >1}// 只保留发现相似/重复的组 (数量 > 1)} }
Step 4: Real-World Usage
// ViewModel 或 Repository 中的实际调用valallPhotos = listOf("/storage/emulated/0/DCIM/Camera/IMG_20260704_001.jpg","/storage/emulated/0/DCIM/Camera/IMG_20260704_002_burst.jpg","/storage/emulated/0/Download/meme.png")// 开启协程在后台线程处理,防止阻塞主线程viewModelScope.launch(Dispatchers.IO) {valduplicateGroups = OpenCVUtils.findDuplicates(allPhotos, threshold =5) duplicateGroups.forEachIndexed { index, group -> Log.d("PapaGallery","发现相似图片组 $index:") group.forEach { path -> Log.d("PapaGallery"," -> $path") } } }
Explore More#opencv-dev
Cross-Platform OpenCV Compilation and Deep Integration with FFmpeg
ArticleA detailed guide on how to compile OpenCV 4.x cleanly using CMake across multiple platforms, resolve core pkg-config linking issues, and link it into FFmpeg to unlock powerful video vision processing filters.
OpenCV Image Duplicate Detection and Similarity Analysis
ArticleA detailed demonstration of how to use OpenCV for image duplicate detection and similarity analysis, including feature extraction, matching algorithms, and result visualization.
Python Calling OpenCV Image Sharpening
ArticleA detailed demonstration of how to use Python to call OpenCV for image sharpening, enhancing image clarity and detail.
C++ Calling OpenCV Image Sharpening
ArticleA detailed demonstration of how to use C++ to call OpenCV for image sharpening, enhancing image clarity and detail.
C++ OpenCV + FFmpeg 7: Making Videos Clearer from Blurry
ArticleA detailed demonstration of how to use C++ and FFmpeg 7 in conjunction with OpenCV for video processing, achieving clarity from blurriness.
