Infrared (IR) and visible image fusion is a crucial technique in data fusion and image processing. It allows for the accurate integration of thermal radiation and texture details from source images. However, current methods often overlook the challenge of high-contrast fusion, resulting in suboptimal performance when replacing thermal radiation target information in IR images with high-contrast information from visible images. To overcome this limitation, we have developed a contrast-balanced framework for IR and visible image fusion. Our innovative approach includes a contrast balance strategy for processing visible images, reducing energy while compensating for overexposed areas in detail. Additionally, a contrast-preserving guided filter decomposes the image into energy-detail layers to filter high contrast and information effectively. To extract active information from the detail layer and brightness information from the energy layer, we introduce Image Statistics technique and a Gaussian distribution of image entropy scheme for fusing the detail and energy layers. The final fused result is achieved by combining the detail and energy layers.Comprehensive experimental results show that our method significantly diminishes contrast issues while maintaining details. Additionally, our approach outperformed leading techniques in both qualitative and quantitative evaluations.