Corrupt JPEG data: 130816 extraneous bytes before marker 0xdb

[realgump]

Search before asking

• I have searched the YOLOv5 issues and found no similar bug report.

YOLOv5 Component

No response

Bug

When training, there is some error occurs:
Corrupt JPEG data: 130816 extraneous bytes before marker 0xdb.

Environment

No response

Minimal Reproducible Example

No response

Additional

There has been a commit to fix that in #4548. However, the errors are raised by opencv-python while the auto-fix is writen by pillow. I found my jpg images pass the pillow verification but still cannot be read by opencv-python, the verify code does nothing but increases time consumption and raise lots of unneccessary warnings. Why not use both opencv-python or pillow in the auto-fix period and the training period?

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[glenn-jocher]

@realgump hi, thank you for your feature suggestion on how to improve YOLOv5 🚀!

The fastest and easiest way to incorporate your ideas into the official codebase is to submit a Pull Request (PR) implementing your idea, and if applicable providing before and after profiling/inference/training results to help us understand the improvement your feature provides. This allows us to directly see the changes in the code and to understand how they affect workflows and performance.

Please see our ✅ Contributing Guide to get started.

[github-actions]

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

• Wiki – https://github.com/ultralytics/yolov5/wiki
• Tutorials – https://docs.ultralytics.com/yolov5
• Docs – https://docs.ultralytics.com

Access additional Ultralytics ⚡ resources:

• Ultralytics HUB – https://ultralytics.com/hub
• Vision API – https://ultralytics.com/yolov5
• About Us – https://ultralytics.com/about
• Join Our Team – https://ultralytics.com/work
• Contact Us – https://ultralytics.com/contact

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

[halqadasi]

We read each JPEG image in the specified folder using OpenCV and overwrite it by re-saving with a quality setting of 95. This re-encoding process fixes any corruption issues by standardizing the image data format.

for filename in tqdm(image_files, desc="Re-encoding images"):

    image_path = os.path.join(folder_path, filename)
    img = cv2.imread(image_path)

    if img is not None:
       # Re-encode and overwrite the image
       cv2.imwrite(image_path, img, [int(cv2.IMWRITE_JPEG_QUALITY), 95])
    else:
        print(f"Failed to read image data: {image_path}")

This solution fixed my problem after long time of debugging.

[pderrenger]

@halqadasi thank you for sharing your solution! Re-encoding images using OpenCV is indeed an effective method to resolve JPEG corruption issues, as it standardizes the format and removes extraneous data. This approach is especially helpful when working with OpenCV for image processing. Your contribution could assist other users facing similar problems—thank you for supporting the YOLO community!